2024-04-25T12:54:02.497 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T12:54:02.497 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460 --owner scheduled_leonidus@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --name leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi -- /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/orig.config.yaml 2024-04-25T12:54:02.556 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T12:54:02.655 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7673460' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4060 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxP6xRdHtzJ2LIVS0OKQ9cP2mrxnzNrXZ753KSRnC5JBbBgyBSE2Dp7PfQiD3aU4KGSHe43+lofbxGK14ZbHUc= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2sD3CpQLsQ+jhRrT4LV86EUUW+/xgHzg5nZ91wd7EZfVT+lsVTpjRHovdL9JP3p42aRAExaXJ/jI43Cd3m0xk= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPADk437WhqxgifqptXf8jpu+bdW+KzofJvFtlOiJKYJ3ZLkZwomBWg8sg1n0cRr1Y6ydR3HmHQsgPObe8ByaL0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_12:17:05 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T12:54:02.655 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa; will attempt to use it 2024-04-25T12:54:02.656 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks 2024-04-25T12:54:02.656 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T12:54:02.657 INFO:teuthology.task.internal:Checking packages... 2024-04-25T12:54:02.680 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T12:54:02.680 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T12:54:02.680 INFO:teuthology.packaging:ref: None 2024-04-25T12:54:02.680 INFO:teuthology.packaging:tag: None 2024-04-25T12:54:02.680 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T12:54:02.680 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:54:02.681 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T12:54:02.924 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T12:54:02.925 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T12:54:02.934 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T12:54:02.935 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T12:54:02.942 INFO:teuthology.task.internal:Saving configuration 2024-04-25T12:54:02.964 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T12:54:02.970 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T12:54:02.996 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:48:13.842521', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxP6xRdHtzJ2LIVS0OKQ9cP2mrxnzNrXZ753KSRnC5JBbBgyBSE2Dp7PfQiD3aU4KGSHe43+lofbxGK14ZbHUc='} 2024-04-25T12:54:03.018 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:48:13.845076', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2sD3CpQLsQ+jhRrT4LV86EUUW+/xgHzg5nZ91wd7EZfVT+lsVTpjRHovdL9JP3p42aRAExaXJ/jI43Cd3m0xk='} 2024-04-25T12:54:03.040 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:48:13.843879', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPADk437WhqxgifqptXf8jpu+bdW+KzofJvFtlOiJKYJ3ZLkZwomBWg8sg1n0cRr1Y6ydR3HmHQsgPObe8ByaL0='} 2024-04-25T12:54:03.040 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T12:54:03.047 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T12:54:03.047 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T12:54:03.047 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T12:54:03.047 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T12:54:03.143 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff4cc34a0d0>, signals=[15]) 2024-04-25T12:54:03.144 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T12:54:03.150 INFO:teuthology.task.internal:Opening connections... 2024-04-25T12:54:03.150 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T12:54:03.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:54:03.232 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-04-25T12:54:03.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:54:03.314 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T12:54:03.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:54:03.385 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T12:54:03.392 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2024-04-25T12:54:03.410 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2024-04-25T12:54:03.410 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="9" 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="9" 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:54:03.469 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:54:03.470 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:54:03.470 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:54:03.470 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:54:03.470 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:54:03.470 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-04-25T12:54:03.496 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-04-25T12:54:03.514 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-04-25T12:54:03.514 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-04-25T12:54:03.571 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-04-25T12:54:03.571 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:54:03.572 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:54:03.572 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-04-25T12:54:03.588 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-04-25T12:54:03.606 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-04-25T12:54:03.606 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-04-25T12:54:03.663 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:54:03.664 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:54:03.665 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:54:03.665 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-04-25T12:54:03.688 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T12:54:03.698 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T12:54:03.704 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T12:54:03.704 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:54:03.707 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:54:03.709 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:54:03.725 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T12:54:03.731 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T12:54:03.731 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:54:03.764 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:54:03.766 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:54:03.880 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T12:54:03.960 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T12:54:03.974 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T12:54:03.974 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T12:54:03.974 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T12:54:03.975 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:54:03.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:54:03.992 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2024-04-25T12:54:04.049 INFO:teuthology.orchestra.run.smithi072.stdout:5.14.0-437.el9.x86_64 2024-04-25T12:54:04.049 INFO:teuthology.task.kernel:Running kernel on smithi072: 5.14.0-437.el9.x86_64 2024-04-25T12:54:04.049 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum install -y kernel 2024-04-25T12:54:07.308 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.0 MB 00:00 2024-04-25T12:54:11.028 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-04-25T12:54:15.304 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-04-25T12:54:16.819 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-04-25T12:54:18.182 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-25T12:54:23.224 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-25T12:54:25.115 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T12:54:25.115 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T12:54:25.235 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-25T12:54:25.236 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-25T12:54:25.236 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-25T12:54:25.364 DEBUG:teuthology.orchestra.run.smithi072:> echo no | sudo yum reinstall kernel || true 2024-04-25T12:54:25.908 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 12:54:23 PM UTC. 2024-04-25T12:54:26.044 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout:Reinstalling: 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 255 k 2024-04-25T12:54:26.070 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 0 2024-04-25T12:54:26.071 INFO:teuthology.orchestra.run.smithi072.stderr:Operation aborted. 2024-04-25T12:54:26.145 INFO:teuthology.orchestra.run.smithi072.stdout:Is this ok [y/N]: 2024-04-25T12:54:26.145 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum reinstall -y kernel || true 2024-04-25T12:54:26.635 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:54:23 PM UTC. 2024-04-25T12:54:26.750 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-25T12:54:26.751 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-25T12:54:26.751 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-04-25T12:54:26.751 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-25T12:54:26.751 INFO:teuthology.orchestra.run.smithi072.stdout:Reinstalling: 2024-04-25T12:54:26.751 INFO:teuthology.orchestra.run.smithi072.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T12:54:26.751 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T12:54:26.751 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-25T12:54:26.752 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-25T12:54:26.752 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T12:54:26.752 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 255 k 2024-04-25T12:54:26.752 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 0 2024-04-25T12:54:26.752 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-25T12:54:27.170 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-25T12:54:27.171 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:54:27.171 INFO:teuthology.orchestra.run.smithi072.stdout:Total 608 kB/s | 255 kB 00:00 2024-04-25T12:54:27.189 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-25T12:54:27.390 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-25T12:54:27.390 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-25T12:54:27.395 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-25T12:54:27.395 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-25T12:54:27.576 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-25T12:54:27.656 INFO:teuthology.orchestra.run.smithi072.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T12:54:27.657 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T12:54:28.672 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T12:54:28.673 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T12:54:29.588 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T12:54:29.588 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T12:54:29.588 INFO:teuthology.orchestra.run.smithi072.stdout:Reinstalled: 2024-04-25T12:54:29.588 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T12:54:29.588 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T12:54:29.588 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-25T12:54:29.800 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T12:54:29.848 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T12:54:29.849 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T12:54:29.849 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T12:54:29.849 DEBUG:teuthology.parallel:result is None 2024-04-25T12:54:29.849 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T12:54:29.855 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T12:54:29.855 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:54:29.857 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:54:29.860 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:54:29.877 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T12:54:29.884 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T12:54:29.890 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T12:54:29.890 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:54:29.913 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:54:29.918 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:54:30.218 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T12:54:30.225 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T12:54:30.225 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:54:30.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:54:30.242 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:54:30.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:54:30.259 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:54:30.261 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:54:30.285 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:54:30.297 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:54:30.317 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:54:30.321 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:54:30.333 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:54:30.353 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:54:30.365 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:54:30.367 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T12:54:30.373 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T12:54:30.374 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:54:30.376 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:54:30.410 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:54:30.438 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T12:54:30.448 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T12:54:30.449 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:54:30.451 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:54:30.477 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:54:30.494 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:54:30.546 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:54:30.599 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:54:30.681 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:54:30.720 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T12:54:30.720 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:54:30.785 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:54:30.829 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:54:30.877 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:54:30.961 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:54:31.001 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T12:54:31.001 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:54:31.067 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:54:31.111 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:54:31.169 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:54:31.254 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:54:31.293 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T12:54:31.293 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:54:31.357 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2024-04-25T12:54:31.360 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-04-25T12:54:31.362 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-04-25T12:54:31.395 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:54:31.396 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:54:31.433 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:54:31.842 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T12:54:31.850 INFO:teuthology.task.internal:Starting timer... 2024-04-25T12:54:31.850 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T12:54:31.859 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T12:54:31.871 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T12:54:31.872 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2024-04-25T12:54:31.938 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2024-04-25T12:54:31.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-04-25T12:54:32.007 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-04-25T12:54:32.009 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-04-25T12:54:32.097 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-04-25T12:54:32.098 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T12:54:32.099 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2024-04-25T12:54:32.119 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2024-04-25T12:54:32.119 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-04-25T12:54:32.146 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-04-25T12:54:32.147 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-04-25T12:54:32.169 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-04-25T12:54:32.170 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi072.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-04-25T12:54:32.170 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:54:32.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:54:32.196 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:54:32.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:54:32.223 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:54:32.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:54:32.247 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T12:54:32.247 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:54:32.281 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:54:32.324 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:54:32.355 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T12:54:32.365 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T12:54:32.380 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T12:54:32.382 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi072.front.sepia.ceph.com,smithi164.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T12:58:24.292 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-04-25T12:58:24.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-25T12:58:24.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:58:24.370 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-04-25T12:58:24.453 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-25T12:58:24.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-25T12:58:24.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:58:24.529 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-25T12:58:24.613 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-25T12:58:24.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-04-25T12:58:24.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:58:24.690 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-04-25T12:58:24.771 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-04-25T12:58:24.772 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T12:58:24.783 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T12:58:24.783 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:58:24.784 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:58:24.786 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:58:24.786 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:58:24.789 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:58:24.789 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:58:24.813 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:58:24.814 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:58:24.827 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:58:24.828 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:58:24.842 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:58:24.850 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-04-25T12:58:24.851 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2024-04-25T12:58:24.856 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:58:24.861 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-04-25T12:58:24.861 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2024-04-25T12:58:24.874 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:58:24.875 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:58:24.880 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-04-25T12:58:24.888 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:58:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:58:24.891 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-04-25T12:58:24.904 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:58:24.917 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:58:24.942 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:58:24.944 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:58:24.946 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:58:24.946 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-04-25T12:58:24.946 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.947 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.947 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.947 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.948 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:58:24.949 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-25T12:58:24.949 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.949 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.949 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.949 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.971 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:58:24.975 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:58:24.975 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-04-25T12:58:24.976 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.976 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.976 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.976 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:58:24.977 INFO:teuthology.run_tasks:Running task install... 2024-04-25T12:58:24.990 DEBUG:teuthology.task.install:project ceph 2024-04-25T12:58:24.990 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T12:58:24.990 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T12:58:24.990 INFO:teuthology.task.install:Using flavor: default 2024-04-25T12:58:25.003 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T12:58:25.004 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T12:58:25.004 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T12:58:25.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:58:25.008 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T12:58:25.008 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:58:25.010 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T12:58:25.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:58:25.155 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T12:58:25.155 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T12:58:25.160 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T12:58:25.160 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T12:58:25.239 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T12:58:25.239 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T12:58:25.397 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:58:25.398 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T12:58:25.398 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:58:25.409 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:58:25.409 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T12:58:25.409 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:58:25.423 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:58:25.423 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:58:25.435 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:58:25.435 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:58:25.488 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:58:25.490 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:58:25.491 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T12:58:25.491 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:58:25.501 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:58:25.516 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:58:25.516 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:58:25.561 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:58:25.575 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:58:25.582 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:58:25.623 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2024-04-25T12:58:25.624 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2024-04-25T12:58:25.638 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-04-25T12:58:25.639 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-04-25T12:58:25.656 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:58:25.720 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-04-25T12:58:25.722 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-04-25T12:58:25.846 INFO:teuthology.orchestra.run.smithi072.stdout:48 files removed 2024-04-25T12:58:25.863 INFO:teuthology.orchestra.run.smithi193.stdout:48 files removed 2024-04-25T12:58:25.873 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:58:25.891 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:58:25.951 INFO:teuthology.orchestra.run.smithi164.stdout:48 files removed 2024-04-25T12:58:25.980 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:58:26.921 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 108 kB/s | 83 kB 00:00 2024-04-25T12:58:26.985 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 98 kB/s | 83 kB 00:00 2024-04-25T12:58:27.329 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 77 kB/s | 83 kB 00:01 2024-04-25T12:58:27.603 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 20 kB/s | 12 kB 00:00 2024-04-25T12:58:27.680 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-04-25T12:58:27.821 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-25T12:58:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-04-25T12:58:28.530 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-04-25T12:58:28.566 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-04-25T12:58:29.333 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T12:58:29.902 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.0 MB 00:01 2024-04-25T12:58:29.983 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-04-25T12:58:31.500 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-25T12:58:32.509 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-25T12:58:34.405 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 19 MB 00:03 2024-04-25T12:58:35.893 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-04-25T12:58:36.718 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T12:58:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-25T12:58:38.233 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-25T12:58:38.497 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T12:58:39.634 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T12:58:39.897 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-25T12:58:41.745 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T12:58:42.201 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 4.6 MB/s | 21 MB 00:04 2024-04-25T12:58:44.617 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T12:58:46.760 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:58:46.761 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:58:46.771 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:58:46.772 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:58:46.773 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:58:46.774 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:58:46.775 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:58:46.777 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:58:46.778 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:58:46.810 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T12:58:46.838 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T12:58:46.857 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-04-25T12:58:46.857 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T12:58:46.857 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T12:58:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T12:58:46.859 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T12:58:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T12:58:46.861 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:58:46.862 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:58:46.863 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T12:58:46.864 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:58:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:58:46.866 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:58:46.867 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:58:46.868 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:58:46.869 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:58:46.869 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:58:46.869 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:58:46.869 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:58:46.869 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:58:46.869 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:58:46.869 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:58:46.869 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:58:46.869 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:58:46.870 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-04-25T12:58:46.870 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:58:46.870 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T12:58:46.870 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T12:58:46.870 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-04-25T12:58:46.870 INFO:teuthology.orchestra.run.smithi193.stdout:Install 148 Packages 2024-04-25T12:58:46.870 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 4 Packages 2024-04-25T12:58:46.870 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T12:58:46.871 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 222 M 2024-04-25T12:58:46.871 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T12:58:47.401 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-25T12:58:48.385 INFO:teuthology.orchestra.run.smithi193.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-25T12:58:48.979 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:58:48.980 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:58:48.990 INFO:teuthology.orchestra.run.smithi072.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:58:48.991 INFO:teuthology.orchestra.run.smithi072.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:58:48.992 INFO:teuthology.orchestra.run.smithi072.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:58:48.992 INFO:teuthology.orchestra.run.smithi072.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:58:48.994 INFO:teuthology.orchestra.run.smithi072.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:58:48.996 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:58:48.997 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:58:49.056 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-25T12:58:49.065 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-04-25T12:58:49.065 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-25T12:58:49.065 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-04-25T12:58:49.066 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-25T12:58:49.066 INFO:teuthology.orchestra.run.smithi072.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:58:49.066 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T12:58:49.066 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:58:49.066 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T12:58:49.066 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T12:58:49.066 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T12:58:49.067 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T12:58:49.067 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T12:58:49.067 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T12:58:49.067 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T12:58:49.067 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T12:58:49.067 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T12:58:49.067 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T12:58:49.067 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T12:58:49.068 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T12:58:49.068 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T12:58:49.068 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T12:58:49.068 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:58:49.068 INFO:teuthology.orchestra.run.smithi072.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:58:49.068 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:58:49.068 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T12:58:49.068 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T12:58:49.069 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T12:58:49.069 INFO:teuthology.orchestra.run.smithi072.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:58:49.069 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:58:49.069 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:58:49.069 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:58:49.069 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:58:49.069 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:58:49.069 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T12:58:49.070 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T12:58:49.070 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T12:58:49.070 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T12:58:49.070 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T12:58:49.070 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:58:49.070 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T12:58:49.070 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:58:49.070 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2024-04-25T12:58:49.070 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T12:58:49.071 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T12:58:49.072 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:58:49.073 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:58:49.074 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:58:49.075 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:58:49.076 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:58:49.077 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:58:49.078 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:58:49.079 INFO:teuthology.orchestra.run.smithi072.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout:Install 148 Packages 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 4 Packages 2024-04-25T12:58:49.080 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T12:58:49.081 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 222 M 2024-04-25T12:58:49.081 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-25T12:58:49.544 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:58:49.544 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:58:49.555 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:58:49.556 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:58:49.556 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:58:49.557 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:58:49.559 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:58:49.561 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:58:49.562 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:58:49.625 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-25T12:58:49.659 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-25T12:58:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-25T12:58:49.659 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-25T12:58:49.659 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-25T12:58:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T12:58:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:58:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-04-25T12:58:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T12:58:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T12:58:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:58:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:58:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:58:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:58:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:58:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:58:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:58:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:58:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:58:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:58:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:58:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:58:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:58:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:58:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:58:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:58:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T12:58:49.673 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-25T12:58:49.673 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-25T12:58:49.673 INFO:teuthology.orchestra.run.smithi164.stdout:Install 148 Packages 2024-04-25T12:58:49.673 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 4 Packages 2024-04-25T12:58:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T12:58:49.673 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 222 M 2024-04-25T12:58:49.674 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-25T12:58:49.752 INFO:teuthology.orchestra.run.smithi193.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 651 kB/s | 890 kB 00:01 2024-04-25T12:58:50.010 INFO:teuthology.orchestra.run.smithi193.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 561 kB/s | 145 kB 00:00 2024-04-25T12:58:50.072 INFO:teuthology.orchestra.run.smithi072.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-25T12:58:50.502 INFO:teuthology.orchestra.run.smithi164.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-25T12:58:50.735 INFO:teuthology.orchestra.run.smithi193.stdout:(4/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 7.9 MB/s | 21 MB 00:02 2024-04-25T12:58:50.830 INFO:teuthology.orchestra.run.smithi072.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 890 kB 00:00 2024-04-25T12:58:50.860 INFO:teuthology.orchestra.run.smithi193.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 12 MB/s | 1.5 MB 00:00 2024-04-25T12:58:50.899 INFO:teuthology.orchestra.run.smithi072.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-25T12:58:51.177 INFO:teuthology.orchestra.run.smithi193.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 15 MB/s | 4.7 MB 00:00 2024-04-25T12:58:52.099 INFO:teuthology.orchestra.run.smithi072.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 2.4 MB 00:01 2024-04-25T12:58:52.525 INFO:teuthology.orchestra.run.smithi072.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.5 MB/s | 1.5 MB 00:00 2024-04-25T12:58:52.561 INFO:teuthology.orchestra.run.smithi193.stdout:(7/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 12 MB/s | 17 MB 00:01 2024-04-25T12:58:53.177 INFO:teuthology.orchestra.run.smithi164.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 333 kB/s | 890 kB 00:02 2024-04-25T12:58:53.442 INFO:teuthology.orchestra.run.smithi072.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.1 MB/s | 4.7 MB 00:00 2024-04-25T12:58:53.444 INFO:teuthology.orchestra.run.smithi193.stdout:(8/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 704 kB/s | 2.4 MB 00:03 2024-04-25T12:58:53.578 INFO:teuthology.orchestra.run.smithi193.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 189 kB/s | 25 kB 00:00 2024-04-25T12:58:53.635 INFO:teuthology.orchestra.run.smithi164.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 316 kB/s | 145 kB 00:00 2024-04-25T12:58:53.945 INFO:teuthology.orchestra.run.smithi193.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 7.7 MB/s | 11 MB 00:01 2024-04-25T12:58:54.028 INFO:teuthology.orchestra.run.smithi193.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.6 MB/s | 225 kB 00:00 2024-04-25T12:58:54.104 INFO:teuthology.orchestra.run.smithi193.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 360 kB/s | 27 kB 00:00 2024-04-25T12:58:54.188 INFO:teuthology.orchestra.run.smithi193.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 378 kB/s | 32 kB 00:00 2024-04-25T12:58:54.346 INFO:teuthology.orchestra.run.smithi193.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.6 MB/s | 747 kB 00:00 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi193.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 163 kB 00:00 2024-04-25T12:58:54.523 INFO:teuthology.orchestra.run.smithi193.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-25T12:58:54.606 INFO:teuthology.orchestra.run.smithi193.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.9 MB/s | 500 kB 00:00 2024-04-25T12:58:55.223 INFO:teuthology.orchestra.run.smithi193.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 8.7 MB/s | 5.4 MB 00:00 2024-04-25T12:58:55.281 INFO:teuthology.orchestra.run.smithi193.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 776 kB/s | 45 kB 00:00 2024-04-25T12:58:55.356 INFO:teuthology.orchestra.run.smithi193.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.8 MB/s | 135 kB 00:00 2024-04-25T12:58:55.432 INFO:teuthology.orchestra.run.smithi193.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.1 MB/s | 161 kB 00:00 2024-04-25T12:58:55.490 INFO:teuthology.orchestra.run.smithi193.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 5.4 MB/s | 320 kB 00:00 2024-04-25T12:58:55.549 INFO:teuthology.orchestra.run.smithi193.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 5.0 MB/s | 298 kB 00:00 2024-04-25T12:58:55.607 INFO:teuthology.orchestra.run.smithi193.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-25T12:58:55.658 INFO:teuthology.orchestra.run.smithi193.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.7 MB/s | 85 kB 00:00 2024-04-25T12:58:55.684 INFO:teuthology.orchestra.run.smithi072.stdout:(7/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 7.5 MB/s | 17 MB 00:02 2024-04-25T12:58:56.033 INFO:teuthology.orchestra.run.smithi193.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 8.2 MB/s | 3.1 MB 00:00 2024-04-25T12:58:56.108 INFO:teuthology.orchestra.run.smithi193.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 170 kB 00:00 2024-04-25T12:58:56.184 INFO:teuthology.orchestra.run.smithi193.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 456 kB/s | 34 kB 00:00 2024-04-25T12:58:56.271 INFO:teuthology.orchestra.run.smithi193.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.6 MB/s | 142 kB 00:00 2024-04-25T12:58:56.759 INFO:teuthology.orchestra.run.smithi193.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.3 MB/s | 3.5 MB 00:00 2024-04-25T12:58:57.110 INFO:teuthology.orchestra.run.smithi164.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 696 kB/s | 2.4 MB 00:03 2024-04-25T12:58:57.168 INFO:teuthology.orchestra.run.smithi072.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 7.1 MB/s | 11 MB 00:01 2024-04-25T12:58:57.243 INFO:teuthology.orchestra.run.smithi072.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 335 kB/s | 25 kB 00:00 2024-04-25T12:58:57.343 INFO:teuthology.orchestra.run.smithi193.stdout:(31/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x 591 kB/s | 5.4 MB 00:09 2024-04-25T12:58:57.535 INFO:teuthology.orchestra.run.smithi193.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-04-25T12:58:57.610 INFO:teuthology.orchestra.run.smithi193.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 659 kB/s | 49 kB 00:00 2024-04-25T12:58:57.685 INFO:teuthology.orchestra.run.smithi193.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 219 kB/s | 16 kB 00:00 2024-04-25T12:58:57.711 INFO:teuthology.orchestra.run.smithi193.stdout:(35/152): ceph-mgr-modules-core-19.0.0-3156.gd6 677 kB/s | 249 kB 00:00 2024-04-25T12:58:57.769 INFO:teuthology.orchestra.run.smithi193.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 273 kB 00:00 2024-04-25T12:58:57.844 INFO:teuthology.orchestra.run.smithi193.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 243 kB/s | 18 kB 00:00 2024-04-25T12:58:57.961 INFO:teuthology.orchestra.run.smithi193.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 690 kB/s | 80 kB 00:00 2024-04-25T12:58:58.030 INFO:teuthology.orchestra.run.smithi193.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 11 MB/s | 803 kB 00:00 2024-04-25T12:58:58.097 INFO:teuthology.orchestra.run.smithi193.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-25T12:58:58.139 INFO:teuthology.orchestra.run.smithi193.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-25T12:58:58.198 INFO:teuthology.orchestra.run.smithi193.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-04-25T12:58:58.231 INFO:teuthology.orchestra.run.smithi193.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T12:58:58.273 INFO:teuthology.orchestra.run.smithi193.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T12:58:58.357 INFO:teuthology.orchestra.run.smithi193.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-25T12:58:58.369 INFO:teuthology.orchestra.run.smithi164.stdout:(5/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 2.6 MB/s | 21 MB 00:08 2024-04-25T12:58:58.491 INFO:teuthology.orchestra.run.smithi193.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-04-25T12:58:58.549 INFO:teuthology.orchestra.run.smithi193.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-04-25T12:58:58.602 INFO:teuthology.orchestra.run.smithi193.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 19 MB/s | 943 kB 00:00 2024-04-25T12:58:58.650 INFO:teuthology.orchestra.run.smithi193.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-25T12:58:58.652 INFO:teuthology.orchestra.run.smithi072.stdout:(10/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x 621 kB/s | 5.4 MB 00:08 2024-04-25T12:58:58.675 INFO:teuthology.orchestra.run.smithi193.stdout:(50/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 790 kB/s | 762 kB 00:00 2024-04-25T12:58:58.693 INFO:teuthology.orchestra.run.smithi164.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 967 kB/s | 1.5 MB 00:01 2024-04-25T12:58:58.700 INFO:teuthology.orchestra.run.smithi193.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-25T12:58:58.726 INFO:teuthology.orchestra.run.smithi193.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T12:58:58.751 INFO:teuthology.orchestra.run.smithi193.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 658 kB/s | 33 kB 00:00 2024-04-25T12:58:58.826 INFO:teuthology.orchestra.run.smithi193.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-25T12:58:58.852 INFO:teuthology.orchestra.run.smithi193.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 187 kB/s | 18 kB 00:00 2024-04-25T12:58:58.877 INFO:teuthology.orchestra.run.smithi193.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-25T12:58:58.902 INFO:teuthology.orchestra.run.smithi193.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T12:58:58.927 INFO:teuthology.orchestra.run.smithi193.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 912 kB/s | 45 kB 00:00 2024-04-25T12:58:58.953 INFO:teuthology.orchestra.run.smithi072.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 749 kB/s | 225 kB 00:00 2024-04-25T12:58:58.953 INFO:teuthology.orchestra.run.smithi193.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T12:58:58.978 INFO:teuthology.orchestra.run.smithi193.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 582 kB 00:00 2024-04-25T12:58:59.003 INFO:teuthology.orchestra.run.smithi193.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 519 kB/s | 25 kB 00:00 2024-04-25T12:58:59.029 INFO:teuthology.orchestra.run.smithi193.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-25T12:58:59.054 INFO:teuthology.orchestra.run.smithi193.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T12:58:59.070 INFO:teuthology.orchestra.run.smithi072.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 231 kB/s | 27 kB 00:00 2024-04-25T12:58:59.086 INFO:teuthology.orchestra.run.smithi164.stdout:(7/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 612 kB/s | 5.4 MB 00:08 2024-04-25T12:58:59.110 INFO:teuthology.orchestra.run.smithi193.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 188 kB 00:00 2024-04-25T12:58:59.137 INFO:teuthology.orchestra.run.smithi193.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 424 kB/s | 35 kB 00:00 2024-04-25T12:58:59.203 INFO:teuthology.orchestra.run.smithi072.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 238 kB/s | 32 kB 00:00 2024-04-25T12:58:59.271 INFO:teuthology.orchestra.run.smithi193.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 34 MB/s | 4.4 MB 00:00 2024-04-25T12:58:59.338 INFO:teuthology.orchestra.run.smithi193.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-25T12:58:59.389 INFO:teuthology.orchestra.run.smithi193.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T12:58:59.422 INFO:teuthology.orchestra.run.smithi193.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T12:58:59.447 INFO:teuthology.orchestra.run.smithi193.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T12:58:59.564 INFO:teuthology.orchestra.run.smithi193.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 41 MB/s | 5.8 MB 00:00 2024-04-25T12:58:59.598 INFO:teuthology.orchestra.run.smithi193.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-04-25T12:58:59.623 INFO:teuthology.orchestra.run.smithi193.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T12:58:59.773 INFO:teuthology.orchestra.run.smithi193.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-04-25T12:58:59.799 INFO:teuthology.orchestra.run.smithi193.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-25T12:58:59.824 INFO:teuthology.orchestra.run.smithi193.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T12:58:59.841 INFO:teuthology.orchestra.run.smithi193.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 618 kB/s | 26 kB 00:00 2024-04-25T12:58:59.858 INFO:teuthology.orchestra.run.smithi193.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 569 kB/s | 19 kB 00:00 2024-04-25T12:58:59.883 INFO:teuthology.orchestra.run.smithi193.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T12:58:59.900 INFO:teuthology.orchestra.run.smithi193.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 838 kB/s | 35 kB 00:00 2024-04-25T12:59:00.017 INFO:teuthology.orchestra.run.smithi193.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-04-25T12:59:00.042 INFO:teuthology.orchestra.run.smithi193.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-25T12:59:00.068 INFO:teuthology.orchestra.run.smithi193.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T12:59:00.093 INFO:teuthology.orchestra.run.smithi193.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T12:59:00.118 INFO:teuthology.orchestra.run.smithi193.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-25T12:59:00.143 INFO:teuthology.orchestra.run.smithi193.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T12:59:00.169 INFO:teuthology.orchestra.run.smithi193.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T12:59:00.194 INFO:teuthology.orchestra.run.smithi193.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T12:59:00.227 INFO:teuthology.orchestra.run.smithi193.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 880 kB/s | 51 kB 00:00 2024-04-25T12:59:00.253 INFO:teuthology.orchestra.run.smithi193.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-04-25T12:59:00.370 INFO:teuthology.orchestra.run.smithi193.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 362 kB/s | 42 kB 00:00 2024-04-25T12:59:00.370 INFO:teuthology.orchestra.run.smithi072.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 640 kB/s | 747 kB 00:01 2024-04-25T12:59:00.587 INFO:teuthology.orchestra.run.smithi193.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-04-25T12:59:00.612 INFO:teuthology.orchestra.run.smithi193.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 184 kB/s | 44 kB 00:00 2024-04-25T12:59:00.637 INFO:teuthology.orchestra.run.smithi193.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T12:59:00.663 INFO:teuthology.orchestra.run.smithi193.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-04-25T12:59:00.669 INFO:teuthology.orchestra.run.smithi164.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 4.7 MB 00:02 2024-04-25T12:59:00.688 INFO:teuthology.orchestra.run.smithi193.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-04-25T12:59:00.713 INFO:teuthology.orchestra.run.smithi193.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T12:59:00.738 INFO:teuthology.orchestra.run.smithi193.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T12:59:00.764 INFO:teuthology.orchestra.run.smithi193.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T12:59:00.789 INFO:teuthology.orchestra.run.smithi193.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T12:59:00.794 INFO:teuthology.orchestra.run.smithi164.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 202 kB/s | 25 kB 00:00 2024-04-25T12:59:00.814 INFO:teuthology.orchestra.run.smithi193.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-25T12:59:00.840 INFO:teuthology.orchestra.run.smithi193.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-04-25T12:59:00.873 INFO:teuthology.orchestra.run.smithi193.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T12:59:01.038 INFO:teuthology.orchestra.run.smithi072.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 245 kB/s | 163 kB 00:00 2024-04-25T12:59:01.307 INFO:teuthology.orchestra.run.smithi193.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 713 kB/s | 309 kB 00:00 2024-04-25T12:59:01.332 INFO:teuthology.orchestra.run.smithi193.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T12:59:01.346 INFO:teuthology.orchestra.run.smithi072.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 410 kB/s | 126 kB 00:00 2024-04-25T12:59:01.382 INFO:teuthology.orchestra.run.smithi193.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-04-25T12:59:01.457 INFO:teuthology.orchestra.run.smithi193.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-04-25T12:59:01.533 INFO:teuthology.orchestra.run.smithi193.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-04-25T12:59:01.616 INFO:teuthology.orchestra.run.smithi193.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T12:59:01.642 INFO:teuthology.orchestra.run.smithi193.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-04-25T12:59:01.692 INFO:teuthology.orchestra.run.smithi193.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-25T12:59:01.767 INFO:teuthology.orchestra.run.smithi193.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-25T12:59:01.801 INFO:teuthology.orchestra.run.smithi193.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-04-25T12:59:01.843 INFO:teuthology.orchestra.run.smithi193.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 578 kB/s | 43 kB 00:00 2024-04-25T12:59:01.868 INFO:teuthology.orchestra.run.smithi193.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-04-25T12:59:01.910 INFO:teuthology.orchestra.run.smithi193.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-25T12:59:01.943 INFO:teuthology.orchestra.run.smithi193.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T12:59:01.985 INFO:teuthology.orchestra.run.smithi193.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-25T12:59:02.027 INFO:teuthology.orchestra.run.smithi193.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-25T12:59:02.052 INFO:teuthology.orchestra.run.smithi193.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 528 kB/s | 35 kB 00:00 2024-04-25T12:59:02.103 INFO:teuthology.orchestra.run.smithi193.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T12:59:02.170 INFO:teuthology.orchestra.run.smithi193.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-25T12:59:02.195 INFO:teuthology.orchestra.run.smithi193.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T12:59:02.229 INFO:teuthology.orchestra.run.smithi193.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-04-25T12:59:02.270 INFO:teuthology.orchestra.run.smithi193.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-25T12:59:02.287 INFO:teuthology.orchestra.run.smithi193.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-25T12:59:02.314 INFO:teuthology.orchestra.run.smithi072.stdout:(17/152): ceph-common-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 21 MB 00:12 2024-04-25T12:59:02.329 INFO:teuthology.orchestra.run.smithi193.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-25T12:59:02.346 INFO:teuthology.orchestra.run.smithi193.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-04-25T12:59:02.347 INFO:teuthology.orchestra.run.smithi072.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 500 kB/s | 500 kB 00:01 2024-04-25T12:59:02.405 INFO:teuthology.orchestra.run.smithi193.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-04-25T12:59:02.447 INFO:teuthology.orchestra.run.smithi193.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-25T12:59:02.472 INFO:teuthology.orchestra.run.smithi193.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-25T12:59:02.514 INFO:teuthology.orchestra.run.smithi193.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-04-25T12:59:02.547 INFO:teuthology.orchestra.run.smithi193.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T12:59:02.581 INFO:teuthology.orchestra.run.smithi193.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-04-25T12:59:02.615 INFO:teuthology.orchestra.run.smithi193.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-25T12:59:02.648 INFO:teuthology.orchestra.run.smithi193.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-04-25T12:59:02.682 INFO:teuthology.orchestra.run.smithi193.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-04-25T12:59:02.732 INFO:teuthology.orchestra.run.smithi193.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T12:59:02.757 INFO:teuthology.orchestra.run.smithi193.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-04-25T12:59:02.799 INFO:teuthology.orchestra.run.smithi193.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 535 kB/s | 36 kB 00:00 2024-04-25T12:59:02.824 INFO:teuthology.orchestra.run.smithi193.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 763 kB/s | 51 kB 00:00 2024-04-25T12:59:02.842 INFO:teuthology.orchestra.run.smithi072.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 91 kB/s | 45 kB 00:00 2024-04-25T12:59:02.892 INFO:teuthology.orchestra.run.smithi193.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-25T12:59:02.917 INFO:teuthology.orchestra.run.smithi193.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 974 kB/s | 90 kB 00:00 2024-04-25T12:59:02.975 INFO:teuthology.orchestra.run.smithi193.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-25T12:59:03.001 INFO:teuthology.orchestra.run.smithi193.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-25T12:59:03.051 INFO:teuthology.orchestra.run.smithi193.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T12:59:03.101 INFO:teuthology.orchestra.run.smithi193.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T12:59:03.209 INFO:teuthology.orchestra.run.smithi072.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 367 kB/s | 135 kB 00:00 2024-04-25T12:59:03.610 INFO:teuthology.orchestra.run.smithi072.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 403 kB/s | 161 kB 00:00 2024-04-25T12:59:04.101 INFO:teuthology.orchestra.run.smithi193.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.1 MB/s | 3.1 MB 00:00 2024-04-25T12:59:04.177 INFO:teuthology.orchestra.run.smithi193.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T12:59:04.285 INFO:teuthology.orchestra.run.smithi193.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-25T12:59:04.877 INFO:teuthology.orchestra.run.smithi072.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 253 kB/s | 320 kB 00:01 2024-04-25T12:59:05.611 INFO:teuthology.orchestra.run.smithi072.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 406 kB/s | 298 kB 00:00 2024-04-25T12:59:05.912 INFO:teuthology.orchestra.run.smithi072.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 331 kB/s | 99 kB 00:00 2024-04-25T12:59:06.120 INFO:teuthology.orchestra.run.smithi072.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 409 kB/s | 85 kB 00:00 2024-04-25T12:59:08.018 INFO:teuthology.orchestra.run.smithi193.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 700 kB/s | 3.4 MB 00:04 2024-04-25T12:59:08.277 INFO:teuthology.orchestra.run.smithi164.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 1.2 MB/s | 11 MB 00:09 2024-04-25T12:59:08.936 INFO:teuthology.orchestra.run.smithi164.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 341 kB/s | 225 kB 00:00 2024-04-25T12:59:09.069 INFO:teuthology.orchestra.run.smithi164.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 203 kB/s | 27 kB 00:00 2024-04-25T12:59:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 253 kB/s | 32 kB 00:00 2024-04-25T12:59:09.579 INFO:teuthology.orchestra.run.smithi072.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 756 kB/s | 5.4 MB 00:07 2024-04-25T12:59:09.896 INFO:teuthology.orchestra.run.smithi072.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 538 kB/s | 170 kB 00:00 2024-04-25T12:59:09.997 INFO:teuthology.orchestra.run.smithi072.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 341 kB/s | 34 kB 00:00 2024-04-25T12:59:10.119 INFO:teuthology.orchestra.run.smithi164.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 807 kB/s | 747 kB 00:00 2024-04-25T12:59:10.289 INFO:teuthology.orchestra.run.smithi072.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 487 kB/s | 142 kB 00:00 2024-04-25T12:59:10.436 INFO:teuthology.orchestra.run.smithi164.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 515 kB/s | 163 kB 00:00 2024-04-25T12:59:10.645 INFO:teuthology.orchestra.run.smithi164.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 608 kB/s | 126 kB 00:00 2024-04-25T12:59:11.395 INFO:teuthology.orchestra.run.smithi164.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 667 kB/s | 500 kB 00:00 2024-04-25T12:59:12.564 INFO:teuthology.orchestra.run.smithi072.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 491 kB/s | 3.1 MB 00:06 2024-04-25T12:59:16.008 INFO:teuthology.orchestra.run.smithi072.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 634 kB/s | 3.5 MB 00:05 2024-04-25T12:59:16.334 INFO:teuthology.orchestra.run.smithi072.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 765 kB/s | 249 kB 00:00 2024-04-25T12:59:16.426 INFO:teuthology.orchestra.run.smithi072.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 537 kB/s | 49 kB 00:00 2024-04-25T12:59:16.518 INFO:teuthology.orchestra.run.smithi072.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 179 kB/s | 16 kB 00:00 2024-04-25T12:59:16.827 INFO:teuthology.orchestra.run.smithi072.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 885 kB/s | 273 kB 00:00 2024-04-25T12:59:18.327 INFO:teuthology.orchestra.run.smithi072.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 508 kB/s | 762 kB 00:01 2024-04-25T12:59:18.428 INFO:teuthology.orchestra.run.smithi072.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 182 kB/s | 18 kB 00:00 2024-04-25T12:59:18.536 INFO:teuthology.orchestra.run.smithi072.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 741 kB/s | 80 kB 00:00 2024-04-25T12:59:18.645 INFO:teuthology.orchestra.run.smithi072.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-04-25T12:59:18.688 INFO:teuthology.orchestra.run.smithi072.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T12:59:18.730 INFO:teuthology.orchestra.run.smithi072.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T12:59:18.753 INFO:teuthology.orchestra.run.smithi164.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 746 kB/s | 5.4 MB 00:07 2024-04-25T12:59:18.814 INFO:teuthology.orchestra.run.smithi072.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-25T12:59:18.856 INFO:teuthology.orchestra.run.smithi072.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T12:59:18.888 INFO:teuthology.orchestra.run.smithi164.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 335 kB/s | 45 kB 00:00 2024-04-25T12:59:18.898 INFO:teuthology.orchestra.run.smithi072.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T12:59:18.934 INFO:teuthology.orchestra.run.smithi072.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.5 MB/s | 125 kB 00:00 2024-04-25T12:59:18.976 INFO:teuthology.orchestra.run.smithi072.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-25T12:59:19.178 INFO:teuthology.orchestra.run.smithi164.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 464 kB/s | 135 kB 00:00 2024-04-25T12:59:19.191 INFO:teuthology.orchestra.run.smithi072.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-04-25T12:59:19.454 INFO:teuthology.orchestra.run.smithi164.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 587 kB/s | 161 kB 00:00 2024-04-25T12:59:19.469 INFO:teuthology.orchestra.run.smithi072.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-04-25T12:59:19.511 INFO:teuthology.orchestra.run.smithi072.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-25T12:59:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 644 kB/s | 21 kB 00:00 2024-04-25T12:59:19.587 INFO:teuthology.orchestra.run.smithi072.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-04-25T12:59:19.621 INFO:teuthology.orchestra.run.smithi072.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 978 kB/s | 33 kB 00:00 2024-04-25T12:59:19.738 INFO:teuthology.orchestra.run.smithi072.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-25T12:59:19.772 INFO:teuthology.orchestra.run.smithi072.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 543 kB/s | 18 kB 00:00 2024-04-25T12:59:19.814 INFO:teuthology.orchestra.run.smithi072.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-04-25T12:59:19.857 INFO:teuthology.orchestra.run.smithi072.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T12:59:19.899 INFO:teuthology.orchestra.run.smithi072.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T12:59:19.949 INFO:teuthology.orchestra.run.smithi072.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T12:59:20.008 INFO:teuthology.orchestra.run.smithi072.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-04-25T12:59:20.044 INFO:teuthology.orchestra.run.smithi072.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 727 kB/s | 25 kB 00:00 2024-04-25T12:59:20.046 INFO:teuthology.orchestra.run.smithi164.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 541 kB/s | 320 kB 00:00 2024-04-25T12:59:20.078 INFO:teuthology.orchestra.run.smithi072.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T12:59:20.120 INFO:teuthology.orchestra.run.smithi072.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-04-25T12:59:20.162 INFO:teuthology.orchestra.run.smithi072.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-25T12:59:20.204 INFO:teuthology.orchestra.run.smithi072.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 839 kB/s | 35 kB 00:00 2024-04-25T12:59:20.388 INFO:teuthology.orchestra.run.smithi072.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-25T12:59:20.539 INFO:teuthology.orchestra.run.smithi072.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-04-25T12:59:20.623 INFO:teuthology.orchestra.run.smithi072.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-25T12:59:20.682 INFO:teuthology.orchestra.run.smithi072.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-04-25T12:59:20.715 INFO:teuthology.orchestra.run.smithi072.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 3.9 MB/s | 133 kB 00:00 2024-04-25T12:59:20.916 INFO:teuthology.orchestra.run.smithi072.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T12:59:20.921 INFO:teuthology.orchestra.run.smithi164.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 340 kB/s | 298 kB 00:00 2024-04-25T12:59:20.975 INFO:teuthology.orchestra.run.smithi072.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-04-25T12:59:21.009 INFO:teuthology.orchestra.run.smithi072.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.5 MB/s | 87 kB 00:00 2024-04-25T12:59:21.209 INFO:teuthology.orchestra.run.smithi072.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-04-25T12:59:21.221 INFO:teuthology.orchestra.run.smithi164.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 331 kB/s | 99 kB 00:00 2024-04-25T12:59:21.253 INFO:teuthology.orchestra.run.smithi072.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.5 MB/s | 244 kB 00:00 2024-04-25T12:59:21.295 INFO:teuthology.orchestra.run.smithi072.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-04-25T12:59:21.329 INFO:teuthology.orchestra.run.smithi072.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 760 kB/s | 26 kB 00:00 2024-04-25T12:59:21.363 INFO:teuthology.orchestra.run.smithi072.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-04-25T12:59:21.405 INFO:teuthology.orchestra.run.smithi072.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T12:59:21.439 INFO:teuthology.orchestra.run.smithi072.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-04-25T12:59:21.555 INFO:teuthology.orchestra.run.smithi164.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 256 kB/s | 85 kB 00:00 2024-04-25T12:59:21.623 INFO:teuthology.orchestra.run.smithi072.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-04-25T12:59:21.666 INFO:teuthology.orchestra.run.smithi072.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-25T12:59:21.708 INFO:teuthology.orchestra.run.smithi072.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-04-25T12:59:21.750 INFO:teuthology.orchestra.run.smithi072.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-25T12:59:21.784 INFO:teuthology.orchestra.run.smithi072.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-04-25T12:59:21.826 INFO:teuthology.orchestra.run.smithi072.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-04-25T12:59:21.868 INFO:teuthology.orchestra.run.smithi072.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-25T12:59:21.911 INFO:teuthology.orchestra.run.smithi072.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-04-25T12:59:21.953 INFO:teuthology.orchestra.run.smithi072.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T12:59:21.996 INFO:teuthology.orchestra.run.smithi072.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-04-25T12:59:22.497 INFO:teuthology.orchestra.run.smithi072.stdout:(90/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-04-25T12:59:22.631 INFO:teuthology.orchestra.run.smithi072.stdout:(91/152): ceph-mgr-diskprediction-local-19.0.0- 751 kB/s | 7.4 MB 00:10 2024-04-25T12:59:22.656 INFO:teuthology.orchestra.run.smithi072.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 262 kB/s | 42 kB 00:00 2024-04-25T12:59:22.682 INFO:teuthology.orchestra.run.smithi072.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 876 kB/s | 44 kB 00:00 2024-04-25T12:59:22.716 INFO:teuthology.orchestra.run.smithi072.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.6 MB/s | 221 kB 00:00 2024-04-25T12:59:22.742 INFO:teuthology.orchestra.run.smithi072.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 611 kB/s | 36 kB 00:00 2024-04-25T12:59:22.767 INFO:teuthology.orchestra.run.smithi072.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 933 kB/s | 47 kB 00:00 2024-04-25T12:59:22.801 INFO:teuthology.orchestra.run.smithi072.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 305 kB 00:00 2024-04-25T12:59:22.827 INFO:teuthology.orchestra.run.smithi072.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-04-25T12:59:22.861 INFO:teuthology.orchestra.run.smithi072.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T12:59:22.886 INFO:teuthology.orchestra.run.smithi072.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T12:59:22.912 INFO:teuthology.orchestra.run.smithi072.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 888 kB/s | 45 kB 00:00 2024-04-25T12:59:22.937 INFO:teuthology.orchestra.run.smithi072.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 442 kB/s | 22 kB 00:00 2024-04-25T12:59:22.979 INFO:teuthology.orchestra.run.smithi072.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T12:59:23.215 INFO:teuthology.orchestra.run.smithi072.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-25T12:59:23.249 INFO:teuthology.orchestra.run.smithi072.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-25T12:59:23.282 INFO:teuthology.orchestra.run.smithi072.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 288 kB/s | 19 kB 00:00 2024-04-25T12:59:23.341 INFO:teuthology.orchestra.run.smithi072.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-04-25T12:59:23.400 INFO:teuthology.orchestra.run.smithi072.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-04-25T12:59:23.492 INFO:teuthology.orchestra.run.smithi072.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T12:59:23.518 INFO:teuthology.orchestra.run.smithi072.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 574 kB/s | 67 kB 00:00 2024-04-25T12:59:23.552 INFO:teuthology.orchestra.run.smithi072.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-25T12:59:23.610 INFO:teuthology.orchestra.run.smithi072.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-04-25T12:59:23.711 INFO:teuthology.orchestra.run.smithi072.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 838 kB 00:00 2024-04-25T12:59:23.737 INFO:teuthology.orchestra.run.smithi072.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 345 kB/s | 43 kB 00:00 2024-04-25T12:59:23.762 INFO:teuthology.orchestra.run.smithi072.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 635 kB/s | 32 kB 00:00 2024-04-25T12:59:23.787 INFO:teuthology.orchestra.run.smithi072.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 296 kB/s | 15 kB 00:00 2024-04-25T12:59:23.822 INFO:teuthology.orchestra.run.smithi072.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.8 MB/s | 172 kB 00:00 2024-04-25T12:59:23.847 INFO:teuthology.orchestra.run.smithi072.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-04-25T12:59:23.891 INFO:teuthology.orchestra.run.smithi072.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.3 MB/s | 303 kB 00:00 2024-04-25T12:59:23.916 INFO:teuthology.orchestra.run.smithi072.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 511 kB/s | 35 kB 00:00 2024-04-25T12:59:23.950 INFO:teuthology.orchestra.run.smithi072.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-25T12:59:24.017 INFO:teuthology.orchestra.run.smithi072.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-04-25T12:59:24.043 INFO:teuthology.orchestra.run.smithi072.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T12:59:24.076 INFO:teuthology.orchestra.run.smithi072.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-04-25T12:59:24.102 INFO:teuthology.orchestra.run.smithi072.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 298 kB/s | 18 kB 00:00 2024-04-25T12:59:24.128 INFO:teuthology.orchestra.run.smithi072.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 452 kB/s | 23 kB 00:00 2024-04-25T12:59:24.154 INFO:teuthology.orchestra.run.smithi072.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 383 kB/s | 19 kB 00:00 2024-04-25T12:59:24.179 INFO:teuthology.orchestra.run.smithi072.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 388 kB/s | 20 kB 00:00 2024-04-25T12:59:24.246 INFO:teuthology.orchestra.run.smithi072.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-25T12:59:24.272 INFO:teuthology.orchestra.run.smithi072.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-04-25T12:59:24.297 INFO:teuthology.orchestra.run.smithi072.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T12:59:24.323 INFO:teuthology.orchestra.run.smithi072.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T12:59:24.366 INFO:teuthology.orchestra.run.smithi072.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.9 MB/s | 272 kB 00:00 2024-04-25T12:59:24.392 INFO:teuthology.orchestra.run.smithi072.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 238 kB/s | 16 kB 00:00 2024-04-25T12:59:24.418 INFO:teuthology.orchestra.run.smithi072.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-25T12:59:24.443 INFO:teuthology.orchestra.run.smithi072.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 556 kB/s | 28 kB 00:00 2024-04-25T12:59:24.469 INFO:teuthology.orchestra.run.smithi072.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-04-25T12:59:24.503 INFO:teuthology.orchestra.run.smithi072.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-25T12:59:24.528 INFO:teuthology.orchestra.run.smithi072.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T12:59:24.554 INFO:teuthology.orchestra.run.smithi072.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 706 kB/s | 36 kB 00:00 2024-04-25T12:59:24.580 INFO:teuthology.orchestra.run.smithi072.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T12:59:24.613 INFO:teuthology.orchestra.run.smithi072.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-25T12:59:24.639 INFO:teuthology.orchestra.run.smithi072.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-25T12:59:24.681 INFO:teuthology.orchestra.run.smithi072.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-04-25T12:59:24.706 INFO:teuthology.orchestra.run.smithi072.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 296 kB/s | 20 kB 00:00 2024-04-25T12:59:24.741 INFO:teuthology.orchestra.run.smithi072.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-25T12:59:24.816 INFO:teuthology.orchestra.run.smithi072.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T12:59:26.413 INFO:teuthology.orchestra.run.smithi164.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 651 kB/s | 3.1 MB 00:04 2024-04-25T12:59:26.713 INFO:teuthology.orchestra.run.smithi164.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 568 kB/s | 170 kB 00:00 2024-04-25T12:59:26.830 INFO:teuthology.orchestra.run.smithi164.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 293 kB/s | 34 kB 00:00 2024-04-25T12:59:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 683 kB/s | 142 kB 00:00 2024-04-25T12:59:28.473 INFO:teuthology.orchestra.run.smithi164.stdout:(30/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 579 kB/s | 17 MB 00:29 2024-04-25T12:59:28.500 INFO:teuthology.orchestra.run.smithi072.stdout:(148/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 1.6 MB/s | 49 MB 00:31 2024-04-25T12:59:28.569 INFO:teuthology.orchestra.run.smithi072.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-25T12:59:28.636 INFO:teuthology.orchestra.run.smithi072.stdout:(150/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 843 kB/s | 3.1 MB 00:03 2024-04-25T12:59:28.703 INFO:teuthology.orchestra.run.smithi072.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-25T12:59:30.233 INFO:teuthology.orchestra.run.smithi164.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 1.1 MB/s | 3.5 MB 00:03 2024-04-25T12:59:30.733 INFO:teuthology.orchestra.run.smithi164.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 498 kB/s | 249 kB 00:00 2024-04-25T12:59:30.833 INFO:teuthology.orchestra.run.smithi164.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 494 kB/s | 49 kB 00:00 2024-04-25T12:59:30.950 INFO:teuthology.orchestra.run.smithi164.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 141 kB/s | 16 kB 00:00 2024-04-25T12:59:31.326 INFO:teuthology.orchestra.run.smithi164.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 729 kB/s | 273 kB 00:00 2024-04-25T12:59:32.067 INFO:teuthology.orchestra.run.smithi164.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.0 MB/s | 762 kB 00:00 2024-04-25T12:59:32.204 INFO:teuthology.orchestra.run.smithi164.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 134 kB/s | 18 kB 00:00 2024-04-25T12:59:32.393 INFO:teuthology.orchestra.run.smithi164.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 426 kB/s | 80 kB 00:00 2024-04-25T12:59:32.526 INFO:teuthology.orchestra.run.smithi164.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.9 MB/s | 803 kB 00:00 2024-04-25T12:59:32.568 INFO:teuthology.orchestra.run.smithi164.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T12:59:32.627 INFO:teuthology.orchestra.run.smithi164.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-25T12:59:32.727 INFO:teuthology.orchestra.run.smithi164.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-25T12:59:32.777 INFO:teuthology.orchestra.run.smithi164.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-25T12:59:32.836 INFO:teuthology.orchestra.run.smithi164.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T12:59:32.878 INFO:teuthology.orchestra.run.smithi164.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T12:59:32.894 INFO:teuthology.orchestra.run.smithi072.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 426 kB/s | 3.4 MB 00:08 2024-04-25T12:59:32.899 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:59:32.899 INFO:teuthology.orchestra.run.smithi072.stdout:Total 5.1 MB/s | 222 MB 00:43 2024-04-25T12:59:32.920 INFO:teuthology.orchestra.run.smithi164.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-04-25T12:59:32.970 INFO:teuthology.orchestra.run.smithi164.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T12:59:33.029 INFO:teuthology.orchestra.run.smithi164.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T12:59:33.071 INFO:teuthology.orchestra.run.smithi164.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-25T12:59:33.138 INFO:teuthology.orchestra.run.smithi164.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 323 kB/s | 21 kB 00:00 2024-04-25T12:59:33.180 INFO:teuthology.orchestra.run.smithi164.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T12:59:33.213 INFO:teuthology.orchestra.run.smithi164.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 986 kB/s | 33 kB 00:00 2024-04-25T12:59:33.339 INFO:teuthology.orchestra.run.smithi164.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-25T12:59:33.389 INFO:teuthology.orchestra.run.smithi164.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-04-25T12:59:33.431 INFO:teuthology.orchestra.run.smithi164.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-04-25T12:59:33.464 INFO:teuthology.orchestra.run.smithi164.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 160 kB 00:00 2024-04-25T12:59:33.498 INFO:teuthology.orchestra.run.smithi164.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T12:59:33.543 INFO:teuthology.orchestra.run.smithi164.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 662 kB 00:00 2024-04-25T12:59:33.582 INFO:teuthology.orchestra.run.smithi164.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 582 kB 00:00 2024-04-25T12:59:33.615 INFO:teuthology.orchestra.run.smithi164.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 763 kB/s | 25 kB 00:00 2024-04-25T12:59:33.649 INFO:teuthology.orchestra.run.smithi164.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T12:59:33.691 INFO:teuthology.orchestra.run.smithi164.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-25T12:59:33.733 INFO:teuthology.orchestra.run.smithi164.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-25T12:59:33.767 INFO:teuthology.orchestra.run.smithi164.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-04-25T12:59:33.909 INFO:teuthology.orchestra.run.smithi164.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 34 MB/s | 4.7 MB 00:00 2024-04-25T12:59:33.966 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-25T12:59:34.059 INFO:teuthology.orchestra.run.smithi164.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 30 MB/s | 4.4 MB 00:00 2024-04-25T12:59:34.108 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-25T12:59:34.108 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-25T12:59:34.134 INFO:teuthology.orchestra.run.smithi164.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-25T12:59:34.185 INFO:teuthology.orchestra.run.smithi164.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 18 MB/s | 911 kB 00:00 2024-04-25T12:59:34.218 INFO:teuthology.orchestra.run.smithi164.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 3.9 MB/s | 133 kB 00:00 2024-04-25T12:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 39 MB/s | 5.8 MB 00:00 2024-04-25T12:59:34.435 INFO:teuthology.orchestra.run.smithi164.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T12:59:34.494 INFO:teuthology.orchestra.run.smithi164.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T12:59:34.711 INFO:teuthology.orchestra.run.smithi164.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 6.0 MB 00:00 2024-04-25T12:59:34.753 INFO:teuthology.orchestra.run.smithi164.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-04-25T12:59:34.795 INFO:teuthology.orchestra.run.smithi164.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-04-25T12:59:34.828 INFO:teuthology.orchestra.run.smithi164.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 774 kB/s | 26 kB 00:00 2024-04-25T12:59:34.854 INFO:teuthology.orchestra.run.smithi164.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 761 kB/s | 19 kB 00:00 2024-04-25T12:59:34.904 INFO:teuthology.orchestra.run.smithi164.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T12:59:34.929 INFO:teuthology.orchestra.run.smithi164.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 1.4 MB/s | 35 kB 00:00 2024-04-25T12:59:35.080 INFO:teuthology.orchestra.run.smithi164.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-04-25T12:59:35.121 INFO:teuthology.orchestra.run.smithi164.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-25T12:59:35.155 INFO:teuthology.orchestra.run.smithi164.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 6.5 MB/s | 222 kB 00:00 2024-04-25T12:59:35.189 INFO:teuthology.orchestra.run.smithi164.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 2.3 MB/s | 77 kB 00:00 2024-04-25T12:59:35.223 INFO:teuthology.orchestra.run.smithi164.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-04-25T12:59:35.265 INFO:teuthology.orchestra.run.smithi164.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-04-25T12:59:35.306 INFO:teuthology.orchestra.run.smithi164.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-25T12:59:35.348 INFO:teuthology.orchestra.run.smithi164.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 6.6 MB/s | 279 kB 00:00 2024-04-25T12:59:35.382 INFO:teuthology.orchestra.run.smithi164.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-04-25T12:59:35.415 INFO:teuthology.orchestra.run.smithi164.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-25T12:59:35.841 INFO:teuthology.orchestra.run.smithi164.stdout:(90/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-25T12:59:35.868 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-25T12:59:35.869 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-25T12:59:35.875 INFO:teuthology.orchestra.run.smithi164.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-04-25T12:59:35.908 INFO:teuthology.orchestra.run.smithi164.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-04-25T12:59:35.951 INFO:teuthology.orchestra.run.smithi164.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-25T12:59:35.985 INFO:teuthology.orchestra.run.smithi164.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-04-25T12:59:36.026 INFO:teuthology.orchestra.run.smithi164.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T12:59:36.068 INFO:teuthology.orchestra.run.smithi164.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 7.2 MB/s | 305 kB 00:00 2024-04-25T12:59:36.102 INFO:teuthology.orchestra.run.smithi164.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-04-25T12:59:36.127 INFO:teuthology.orchestra.run.smithi164.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 3.7 MB/s | 94 kB 00:00 2024-04-25T12:59:36.160 INFO:teuthology.orchestra.run.smithi164.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-04-25T12:59:36.194 INFO:teuthology.orchestra.run.smithi164.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T12:59:36.228 INFO:teuthology.orchestra.run.smithi164.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 672 kB/s | 22 kB 00:00 2024-04-25T12:59:36.278 INFO:teuthology.orchestra.run.smithi164.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 17 MB/s | 863 kB 00:00 2024-04-25T12:59:36.745 INFO:teuthology.orchestra.run.smithi164.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T12:59:36.845 INFO:teuthology.orchestra.run.smithi164.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-04-25T12:59:36.920 INFO:teuthology.orchestra.run.smithi164.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-25T12:59:37.195 INFO:teuthology.orchestra.run.smithi164.stdout:(106/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T12:59:37.270 INFO:teuthology.orchestra.run.smithi164.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 348 kB/s | 26 kB 00:00 2024-04-25T12:59:37.346 INFO:teuthology.orchestra.run.smithi164.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-04-25T12:59:37.421 INFO:teuthology.orchestra.run.smithi164.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 898 kB/s | 67 kB 00:00 2024-04-25T12:59:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-25T12:59:37.588 INFO:teuthology.orchestra.run.smithi164.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-25T12:59:37.664 INFO:teuthology.orchestra.run.smithi164.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 7.2 MB/s | 548 kB 00:00 2024-04-25T12:59:37.739 INFO:teuthology.orchestra.run.smithi164.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 579 kB/s | 43 kB 00:00 2024-04-25T12:59:37.814 INFO:teuthology.orchestra.run.smithi164.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 430 kB/s | 32 kB 00:00 2024-04-25T12:59:37.889 INFO:teuthology.orchestra.run.smithi164.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2024-04-25T12:59:37.965 INFO:teuthology.orchestra.run.smithi164.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T12:59:38.040 INFO:teuthology.orchestra.run.smithi164.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-25T12:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-25T12:59:38.191 INFO:teuthology.orchestra.run.smithi164.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 471 kB/s | 35 kB 00:00 2024-04-25T12:59:38.266 INFO:teuthology.orchestra.run.smithi164.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T12:59:38.366 INFO:teuthology.orchestra.run.smithi164.stdout:(121/152): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-04-25T12:59:38.442 INFO:teuthology.orchestra.run.smithi164.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-25T12:59:38.508 INFO:teuthology.orchestra.run.smithi164.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-04-25T12:59:38.575 INFO:teuthology.orchestra.run.smithi164.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-04-25T12:59:38.642 INFO:teuthology.orchestra.run.smithi164.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-04-25T12:59:38.709 INFO:teuthology.orchestra.run.smithi164.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 292 kB/s | 19 kB 00:00 2024-04-25T12:59:38.776 INFO:teuthology.orchestra.run.smithi164.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 295 kB/s | 20 kB 00:00 2024-04-25T12:59:38.876 INFO:teuthology.orchestra.run.smithi164.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-25T12:59:38.943 INFO:teuthology.orchestra.run.smithi164.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 694 kB/s | 46 kB 00:00 2024-04-25T12:59:39.019 INFO:teuthology.orchestra.run.smithi164.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-25T12:59:39.086 INFO:teuthology.orchestra.run.smithi164.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 867 kB/s | 58 kB 00:00 2024-04-25T12:59:39.161 INFO:teuthology.orchestra.run.smithi164.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-04-25T12:59:39.228 INFO:teuthology.orchestra.run.smithi164.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-04-25T12:59:39.303 INFO:teuthology.orchestra.run.smithi164.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T12:59:39.428 INFO:teuthology.orchestra.run.smithi164.stdout:(135/152): ceph-mgr-diskprediction-local-19.0.0 690 kB/s | 7.4 MB 00:10 2024-04-25T12:59:39.454 INFO:teuthology.orchestra.run.smithi164.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 188 kB/s | 28 kB 00:00 2024-04-25T12:59:39.529 INFO:teuthology.orchestra.run.smithi164.stdout:(137/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-25T12:59:39.604 INFO:teuthology.orchestra.run.smithi164.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 790 kB/s | 59 kB 00:00 2024-04-25T12:59:39.679 INFO:teuthology.orchestra.run.smithi164.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 477 kB/s | 36 kB 00:00 2024-04-25T12:59:39.705 INFO:teuthology.orchestra.run.smithi164.stdout:(140/152): python3-repoze-lru-0.7-16.el9.noarch 112 kB/s | 31 kB 00:00 2024-04-25T12:59:39.755 INFO:teuthology.orchestra.run.smithi164.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 680 kB/s | 51 kB 00:00 2024-04-25T12:59:39.822 INFO:teuthology.orchestra.run.smithi164.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-25T12:59:39.897 INFO:teuthology.orchestra.run.smithi164.stdout:(143/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-25T12:59:39.922 INFO:teuthology.orchestra.run.smithi164.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2024-04-25T12:59:39.972 INFO:teuthology.orchestra.run.smithi164.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-25T12:59:40.023 INFO:teuthology.orchestra.run.smithi164.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T12:59:40.073 INFO:teuthology.orchestra.run.smithi164.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T12:59:41.833 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-25T12:59:41.942 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:59:41.989 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:59:42.051 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:59:42.096 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:59:42.130 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:59:42.173 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:59:42.502 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:59:42.692 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:59:42.798 INFO:teuthology.orchestra.run.smithi164.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 3.1 MB 00:02 2024-04-25T12:59:42.813 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:59:42.845 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:59:42.873 INFO:teuthology.orchestra.run.smithi164.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 19 MB/s | 1.3 MB 00:00 2024-04-25T12:59:42.937 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:59:42.975 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:59:42.990 INFO:teuthology.orchestra.run.smithi164.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-04-25T12:59:43.128 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T12:59:43.183 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T12:59:43.213 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:59:43.299 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:59:43.324 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:59:43.378 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:59:43.400 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:59:43.425 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:59:43.626 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:59:43.662 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:59:43.716 INFO:teuthology.orchestra.run.smithi164.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 942 kB/s | 3.4 MB 00:03 2024-04-25T12:59:43.770 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T12:59:43.866 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T12:59:43.905 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T12:59:43.940 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T12:59:44.014 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T12:59:44.057 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T12:59:44.102 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T12:59:44.161 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T12:59:44.198 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T12:59:44.275 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T12:59:44.330 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T12:59:44.361 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T12:59:44.406 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T12:59:44.483 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T12:59:44.518 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T12:59:44.562 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T12:59:44.591 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T12:59:44.663 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T12:59:44.715 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T12:59:44.791 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T12:59:44.938 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T12:59:44.997 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T12:59:45.532 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T12:59:45.630 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T12:59:45.668 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T12:59:45.711 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T12:59:45.750 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T12:59:45.792 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T12:59:45.995 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T12:59:46.049 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T12:59:46.105 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T12:59:46.187 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T12:59:46.229 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T12:59:46.264 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T12:59:46.313 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T12:59:46.349 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T12:59:46.393 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T12:59:46.445 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T12:59:46.582 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T12:59:46.637 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T12:59:46.668 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T12:59:46.718 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T12:59:46.813 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T12:59:46.846 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T12:59:46.885 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T12:59:46.922 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T12:59:46.955 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T12:59:47.039 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T12:59:47.179 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T12:59:47.227 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T12:59:47.264 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T12:59:47.299 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T12:59:47.339 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T12:59:47.371 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T12:59:47.403 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T12:59:47.438 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T12:59:47.472 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T12:59:47.514 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T12:59:47.549 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T12:59:47.637 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T12:59:47.690 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T12:59:47.731 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T12:59:47.779 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T12:59:47.886 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T12:59:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T12:59:48.686 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T12:59:48.745 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T12:59:48.780 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T12:59:48.883 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T12:59:48.941 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T12:59:49.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T12:59:49.069 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T12:59:49.248 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T12:59:49.510 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T12:59:49.739 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T12:59:49.848 INFO:teuthology.orchestra.run.smithi164.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 1.0 MB/s | 49 MB 00:49 2024-04-25T12:59:49.854 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:59:49.854 INFO:teuthology.orchestra.run.smithi164.stdout:Total 3.7 MB/s | 222 MB 01:00 2024-04-25T12:59:50.391 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T12:59:50.526 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T12:59:50.963 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-25T12:59:51.108 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-25T12:59:51.108 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-25T12:59:51.756 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T12:59:51.864 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T12:59:51.901 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T12:59:51.935 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T12:59:52.248 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T12:59:52.307 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:59:52.400 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:59:52.431 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T12:59:52.468 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T12:59:52.877 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-25T12:59:52.878 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-25T12:59:52.947 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T12:59:52.981 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:59:53.053 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:59:53.084 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T12:59:54.329 INFO:teuthology.orchestra.run.smithi193.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 827 kB/s | 49 MB 01:00 2024-04-25T12:59:54.334 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:59:54.335 INFO:teuthology.orchestra.run.smithi193.stdout:Total 3.3 MB/s | 222 MB 01:07 2024-04-25T12:59:55.374 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:59:55.381 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T12:59:55.422 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:59:55.523 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T12:59:55.523 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T12:59:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:59:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:59:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:59:56.116 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T12:59:56.116 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T12:59:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T12:59:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:59:57.469 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T12:59:57.470 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T12:59:58.791 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-25T12:59:58.877 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:59:58.915 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:59:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:59:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:59:59.069 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:59:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:59:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:59:59.545 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:59:59.674 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:59:59.730 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:59:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:59:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:00:00.002 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:00:00.047 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:00:00.075 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:00:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:00:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:00:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:00:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:00:00.318 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:00:00.510 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:00:00.544 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:00:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:00:00.739 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:00:00.776 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:00:00.811 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:00:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:00:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:00:00.970 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:00:01.002 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:00:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:00:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:00:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:00:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:00:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:00:01.286 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:00:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T13:00:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T13:00:01.393 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T13:00:01.464 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:00:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:00:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:00:01.753 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:00:01.810 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:00:02.339 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:00:02.431 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:00:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:00:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:00:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T13:00:02.589 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:00:02.751 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:00:02.801 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:00:02.857 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:00:02.929 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:00:02.959 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:00:02.996 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:00:03.043 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:00:03.078 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:00:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:00:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:00:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:00:03.353 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:00:03.382 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:00:03.424 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:00:03.425 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T13:00:03.516 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:00:03.519 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:00:03.554 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:00:03.565 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:00:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:00:03.607 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:00:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:00:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:00:03.658 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:00:03.684 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:00:03.726 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:00:03.731 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:00:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:00:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:00:03.924 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:00:03.958 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:00:03.997 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:00:04.029 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:00:04.047 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:00:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:00:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:00:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:00:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:00:04.168 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:00:04.203 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:00:04.273 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:00:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:00:04.294 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:00:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:00:04.373 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:00:04.384 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:00:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:00:04.420 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:00:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:00:04.564 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:00:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:00:04.612 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:00:04.640 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:00:04.727 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:00:04.749 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:00:04.803 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:00:04.825 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:00:04.854 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:00:05.042 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:00:05.075 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:00:05.181 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:00:05.271 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:00:05.301 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:00:05.316 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:00:05.346 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:00:05.351 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:00:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:00:05.425 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:00:05.459 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:00:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:00:05.502 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:00:05.534 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:00:05.538 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:00:05.562 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:00:05.624 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:00:05.634 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:00:05.682 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:00:05.692 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:00:05.723 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:00:05.767 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:00:05.843 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:00:05.849 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:00:05.878 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T13:00:05.879 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:00:05.921 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T13:00:05.950 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T13:00:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:00:06.018 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:00:06.073 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:00:06.146 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:00:06.279 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:00:06.333 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:00:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:00:06.745 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:00:06.855 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:00:06.936 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:00:06.967 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:00:07.009 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:00:07.047 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T13:00:07.080 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:00:07.226 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:00:07.275 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:00:07.329 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:00:07.394 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:00:07.426 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:00:07.462 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:00:07.508 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:00:07.545 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:00:07.588 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:00:07.630 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:00:07.765 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:00:07.810 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:00:07.840 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:00:07.882 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:00:07.976 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:00:07.996 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:00:08.007 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:00:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:00:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:00:08.089 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:00:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:00:08.119 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:00:08.152 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:00:08.163 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:00:08.290 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:00:08.321 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:00:08.349 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:00:08.383 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:00:08.422 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:00:08.454 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:00:08.463 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:00:08.484 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:00:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:00:08.518 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:00:08.552 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:00:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:00:08.593 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:00:08.611 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T13:00:08.628 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:00:08.647 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:00:08.715 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:00:08.758 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:00:08.790 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:00:08.828 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:00:08.927 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:00:08.983 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:00:09.127 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:00:09.167 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:00:09.239 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:00:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T13:00:09.704 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:00:09.755 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:00:09.788 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:00:09.887 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:00:09.938 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:00:10.025 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:00:10.057 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:00:10.233 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:00:10.263 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:00:10.340 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:00:10.773 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:00:10.773 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2024-04-25T13:00:10.773 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2024-04-25T13:00:10.773 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2024-04-25T13:00:10.773 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2024-04-25T13:00:10.773 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2024-04-25T13:00:10.773 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /root 2024-04-25T13:00:10.773 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2024-04-25T13:00:10.773 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:00:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:00:10.964 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:10.964 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:00:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:10.973 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:00:11.085 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:00:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:00:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:00:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:00:12.152 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:00:12.244 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:00:12.245 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:00:12.245 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:00:12.245 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:00:12.300 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:00:12.383 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:00:12.412 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:00:12.445 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:00:12.472 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:00:12.498 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:00:12.498 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:12.498 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:00:12.498 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:00:12.498 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:00:12.498 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:12.736 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:00:12.755 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:00:12.755 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:12.755 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:00:12.755 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:00:12.756 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:00:12.756 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:12.774 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:00:12.816 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:00:12.905 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:00:12.937 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T13:00:12.973 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:00:13.201 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:00:13.224 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:00:13.224 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:13.224 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:00:13.224 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:00:13.224 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:00:13.225 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:13.334 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:00:13.391 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:00:13.452 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:00:13.455 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:00:13.485 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:00:13.556 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:00:13.578 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T13:00:13.641 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:00:13.693 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:00:13.771 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:00:13.869 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:00:14.010 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:00:14.054 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:00:14.087 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:00:14.117 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:00:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:00:14.613 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:00:14.645 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T13:00:14.708 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T13:00:14.745 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:00:15.785 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:00:15.824 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:00:15.832 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:00:15.866 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:00:16.404 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:00:16.445 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:00:16.529 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:00:16.529 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:00:16.530 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:00:16.530 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:16.554 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:00:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:00:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:00:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:00:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T13:00:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:00:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:00:17.144 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:17.144 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:00:17.144 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:00:17.144 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:00:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:00:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:00:17.298 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T13:00:17.347 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T13:00:18.268 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:00:18.304 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:00:18.326 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:00:18.327 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:18.327 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:00:18.327 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:00:18.327 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:00:18.327 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:18.372 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:00:18.394 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:00:18.394 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:18.394 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:00:18.394 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:18.449 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:00:18.471 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:00:18.471 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:18.471 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:00:18.471 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:00:18.471 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:00:18.471 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:18.775 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:00:18.797 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:00:18.797 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:18.797 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:00:18.797 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:00:18.797 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:00:18.797 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:23.401 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T13:00:23.443 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T13:00:23.484 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T13:00:23.518 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:00:23.554 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:00:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:00:23.593 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-04-25T13:00:23.593 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-04-25T13:00:23.593 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-04-25T13:00:23.593 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-04-25T13:00:23.593 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-04-25T13:00:23.593 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-04-25T13:00:23.593 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-04-25T13:00:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:23.731 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T13:00:23.773 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:00:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:00:23.789 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:23.789 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:00:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:23.790 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:00:23.820 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T13:00:24.448 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:00:24.500 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:00:24.520 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:00:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:00:25.293 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:00:25.294 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:25.294 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:00:25.294 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:00:25.294 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:00:25.294 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:25.373 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T13:00:25.478 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:00:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:00:25.528 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:00:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:00:25.541 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:25.541 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:00:25.541 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:00:25.541 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:00:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:25.621 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:00:25.654 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:00:25.654 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:00:25.747 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:00:25.781 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:00:25.781 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:00:25.878 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:00:25.948 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:00:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:00:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:00:26.009 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:26.009 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:00:26.009 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:00:26.009 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:00:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:26.116 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:00:26.189 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:00:26.245 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:00:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:00:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:00:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:00:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:00:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:00:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:00:26.852 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:00:26.879 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:00:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:00:27.353 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:00:27.385 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T13:00:27.447 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T13:00:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:00:28.036 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:00:28.036 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-04-25T13:00:28.036 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-04-25T13:00:28.036 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-04-25T13:00:28.036 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-04-25T13:00:28.036 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-04-25T13:00:28.036 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-04-25T13:00:28.037 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-04-25T13:00:28.037 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:28.208 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:00:28.221 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:00:28.221 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:28.221 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:00:28.221 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:00:28.561 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:00:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:00:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:00:29.672 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:00:29.717 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:00:29.742 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:00:29.742 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:29.743 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:00:29.743 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:00:29.743 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:00:29.743 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T13:00:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:00:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:00:29.912 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:29.912 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:00:29.912 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:00:29.912 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:00:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:29.956 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:00:29.967 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:00:29.968 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:00:29.991 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:00:29.991 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:29.991 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:00:29.991 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:00:29.991 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:00:29.991 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:30.074 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T13:00:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T13:00:30.444 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:00:30.466 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:00:30.466 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:30.466 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:00:30.466 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:00:30.466 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:00:30.467 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:30.583 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:00:30.640 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:00:30.695 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:00:30.880 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:00:30.936 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:00:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:00:31.055 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:00:31.083 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:00:31.103 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:00:31.106 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:00:31.107 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:31.107 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:00:31.107 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:00:31.107 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:00:31.107 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:31.151 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:00:31.173 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:00:31.173 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:31.173 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:00:31.173 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:00:31.240 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:00:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:00:31.249 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:31.249 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:00:31.249 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:00:31.249 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:00:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:31.286 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:00:31.318 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:00:31.346 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:00:31.552 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:00:31.574 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:00:31.574 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:31.575 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:00:31.575 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:00:31.575 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:00:31.575 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:31.798 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:00:31.819 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:00:31.851 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T13:00:31.913 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T13:00:31.943 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:00:32.978 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:00:33.010 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:00:33.970 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:00:33.994 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:00:34.095 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:00:34.258 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T13:00:34.288 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:00:34.310 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:00:34.311 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:34.311 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:00:34.311 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:00:34.311 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:00:34.311 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:34.355 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:00:34.366 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:00:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T13:00:34.512 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T13:00:35.429 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:00:35.458 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:00:35.480 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:00:35.481 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:35.481 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:00:35.481 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:00:35.481 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:00:35.481 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:35.525 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:00:35.546 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:00:35.547 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:35.547 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:00:35.547 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:35.592 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:00:35.614 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:00:35.614 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:35.614 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:00:35.614 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:00:35.614 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:00:35.614 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:35.918 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:00:35.940 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:00:35.940 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:00:35.940 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:00:35.940 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:00:35.940 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:00:35.941 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T13:00:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T13:00:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T13:00:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:00:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:00:36.329 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:00:36.329 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T13:00:36.329 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T13:00:36.329 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T13:00:36.329 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T13:00:36.330 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T13:00:36.330 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T13:00:36.330 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T13:00:36.330 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:00:36.330 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:00:36.331 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:00:36.331 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T13:00:36.331 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T13:00:36.331 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T13:00:36.332 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T13:00:36.333 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T13:00:36.333 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T13:00:36.346 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T13:00:36.346 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T13:00:36.346 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T13:00:36.346 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T13:00:36.346 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T13:00:36.346 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T13:00:36.346 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T13:00:36.346 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:00:36.347 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:00:36.348 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:00:36.349 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:00:36.350 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:00:36.351 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:00:36.352 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:00:36.352 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:00:36.352 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:00:36.352 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:00:36.352 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:00:36.352 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:00:36.352 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:00:36.352 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:00:36.353 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:00:36.354 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:00:36.354 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:00:36.354 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:00:36.354 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:00:36.354 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:00:36.354 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:00:36.354 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:00:36.354 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:00:36.355 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:00:36.356 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:00:36.356 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:00:36.356 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:00:36.356 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:00:36.356 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:00:36.356 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:00:36.356 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:00:36.357 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:00:36.357 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:00:36.357 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T13:00:36.357 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:00:36.357 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T13:00:36.357 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:00:36.357 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:00:36.358 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:00:36.358 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:00:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T13:00:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:00:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T13:00:37.156 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:00:37.195 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:00:37.215 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:00:38.066 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T13:00:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:00:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:00:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:00:38.312 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:00:38.312 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:00:38.404 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:00:38.437 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:00:38.438 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:00:38.525 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:00:38.594 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:00:38.673 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:00:38.674 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.675 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:00:38.676 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:00:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:00:38.678 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:00:38.678 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:00:38.678 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.678 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.678 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:00:38.678 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.678 INFO:teuthology.orchestra.run.smithi072.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:00:38.678 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:00:38.678 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:00:38.679 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:00:38.680 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:00:38.681 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:00:38.682 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:00:38.683 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:38.684 INFO:teuthology.orchestra.run.smithi072.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:00:38.685 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:00:38.685 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:00:38.685 INFO:teuthology.orchestra.run.smithi072.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:00:38.685 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:00:38.685 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:00:38.685 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:00:38.685 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:00:38.685 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-25T13:00:39.176 DEBUG:teuthology.parallel:result is None 2024-04-25T13:00:40.538 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T13:00:40.579 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T13:00:40.620 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T13:00:40.652 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:00:40.679 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:00:40.863 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T13:00:40.908 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:00:40.938 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T13:00:41.559 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:00:41.602 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:00:41.622 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:00:42.443 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T13:00:42.532 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:00:42.572 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:00:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:00:42.693 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:00:42.694 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:00:42.761 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:00:42.794 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:00:42.794 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:00:42.864 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:00:42.934 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:00:44.409 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:00:44.409 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T13:00:44.409 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T13:00:44.409 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T13:00:44.409 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T13:00:44.409 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T13:00:44.409 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T13:00:44.409 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T13:00:44.409 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T13:00:44.410 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T13:00:44.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T13:00:44.427 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T13:00:44.427 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T13:00:44.427 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T13:00:44.427 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T13:00:44.427 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T13:00:44.427 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T13:00:44.427 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T13:00:44.428 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T13:00:44.428 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T13:00:44.428 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T13:00:44.428 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T13:00:44.428 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:00:44.428 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:00:44.428 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:00:44.428 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:00:44.429 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:00:44.429 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:00:44.429 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:00:44.429 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:00:44.429 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:00:44.429 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:00:44.429 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:00:44.429 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:00:44.430 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:00:44.430 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:00:44.430 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:00:44.430 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:00:44.430 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:00:44.430 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:00:44.430 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:00:44.430 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:00:44.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:00:44.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:00:44.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:00:44.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:00:44.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:00:44.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:00:44.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:00:44.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:00:44.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:00:44.432 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:00:44.432 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:00:44.432 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:00:44.432 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:00:44.432 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:00:44.432 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:00:44.432 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:00:44.432 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:00:44.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:00:44.434 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:00:44.435 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:00:44.436 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:00:44.437 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:00:44.438 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:00:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:00:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:00:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:00:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:00:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:00:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:00:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:00:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:00:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:00:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:00:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:00:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:00:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:00:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T13:00:46.592 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-25T13:00:46.955 DEBUG:teuthology.parallel:result is None 2024-04-25T13:00:48.527 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:00:48.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T13:00:48.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T13:00:48.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T13:00:48.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T13:00:48.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T13:00:48.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T13:00:48.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T13:00:48.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:00:48.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:00:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:00:48.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:00:48.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:00:48.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T13:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T13:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:00:50.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:00:50.587 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:50.587 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-04-25T13:00:50.587 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.587 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:00:50.590 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.591 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:00:50.592 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:00:50.593 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:00:50.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:00:50.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:00:50.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:00:50.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:00:50.598 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:00:50.599 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:00:50.599 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:00:50.599 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:00:50.599 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:00:50.599 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:00:50.599 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:00:50.599 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:00:50.600 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T13:00:51.008 DEBUG:teuthology.parallel:result is None 2024-04-25T13:00:51.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:00:51.301 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:00:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T13:00:51.328 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T13:00:51.328 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T13:00:51.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:00:51.577 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:00:51.603 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T13:00:51.603 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T13:00:51.603 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T13:00:51.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:00:51.832 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:00:51.854 INFO:teuthology.orchestra.run.smithi193.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T13:00:51.854 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T13:00:51.854 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T13:00:51.856 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T13:00:51.856 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:00:51.856 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:00:51.887 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T13:00:51.887 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:00:51.918 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T13:00:51.918 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:00:51.944 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T13:00:51.944 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:00:51.944 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:00:51.971 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:00:52.037 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T13:00:52.037 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:00:52.065 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:00:52.132 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T13:00:52.133 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:00:52.156 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:00:52.218 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T13:00:52.219 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:00:52.219 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:00:52.244 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:00:52.310 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T13:00:52.310 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:00:52.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:00:52.406 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T13:00:52.406 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:00:52.430 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:00:52.494 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T13:00:52.494 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:00:52.494 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:00:52.520 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:00:52.585 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T13:00:52.585 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:00:52.613 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:00:52.682 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T13:00:52.682 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:00:52.706 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:00:52.769 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T13:00:52.868 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T13:00:52.869 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:00:52.869 INFO:tasks.cephadm:Cluster fsid is d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:00:52.869 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T13:00:52.869 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.72', 'mon.b': '172.21.15.164', 'mon.c': '172.21.15.193'} 2024-04-25T13:00:52.869 INFO:tasks.cephadm:First mon is mon.a on smithi072 2024-04-25T13:00:52.870 INFO:tasks.cephadm:First mgr is x 2024-04-25T13:00:52.870 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T13:00:52.870 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname $(hostname -s) 2024-04-25T13:00:52.896 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-04-25T13:00:52.925 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-04-25T13:00:52.948 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T13:00:52.948 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:00:53.170 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T13:00:53.316 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:00:53.317 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T13:00:53.317 DEBUG:teuthology.orchestra.run.smithi072:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:00:54.532 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 13:00 /home/ubuntu/cephtest/cephadm 2024-04-25T13:00:54.533 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:00:55.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 13:00 /home/ubuntu/cephtest/cephadm 2024-04-25T13:00:55.396 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:00:56.275 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 13:00 /home/ubuntu/cephtest/cephadm 2024-04-25T13:00:56.275 DEBUG:teuthology.orchestra.run.smithi072:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:00:56.294 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:00:56.313 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:00:56.350 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T13:00:56.351 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T13:00:56.353 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T13:00:56.355 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T13:00:57.962 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:00:58.078 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:00:59.249 INFO:teuthology.orchestra.run.smithi072.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:02:47.003 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T13:02:47.003 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T13:02:47.003 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T13:02:47.003 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-04-25T13:02:47.003 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T13:02:47.004 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-04-25T13:02:47.004 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T13:02:47.363 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:02:47.363 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T13:02:47.363 INFO:teuthology.orchestra.run.smithi072.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T13:02:47.363 INFO:teuthology.orchestra.run.smithi072.stdout: "repo_digests": [ 2024-04-25T13:02:47.363 INFO:teuthology.orchestra.run.smithi072.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T13:02:47.363 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:02:47.363 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:02:48.749 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T13:02:48.749 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T13:02:48.749 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T13:02:48.749 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-04-25T13:02:48.749 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T13:02:48.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-04-25T13:02:48.749 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T13:02:48.776 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph 2024-04-25T13:02:48.808 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-04-25T13:02:48.836 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-04-25T13:02:48.870 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /etc/ceph 2024-04-25T13:02:48.897 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-04-25T13:02:48.921 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-04-25T13:02:48.949 INFO:tasks.cephadm:Writing seed config... 2024-04-25T13:02:48.950 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T13:02:48.950 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T13:02:48.950 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T13:02:48.950 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T13:02:48.950 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T13:02:48.950 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T13:02:48.950 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T13:02:48.951 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T13:02:48.952 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T13:02:48.953 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T13:02:48.954 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T13:02:48.954 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T13:02:48.954 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T13:02:48.954 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:02:48.954 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T13:02:48.971 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d8ca43ea-0303-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T13:02:48.971 DEBUG:teuthology.orchestra.run.smithi072:mon.a> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T13:02:49.012 DEBUG:teuthology.orchestra.run.smithi072:mgr.x> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T13:02:49.055 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T13:02:49.056 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.72 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:02:49.246 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:02:49.247 INFO:teuthology.orchestra.run.smithi072.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'd8ca43ea-0303-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.72', '--skip-admin-label'] 2024-04-25T13:02:49.247 INFO:teuthology.orchestra.run.smithi072.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T13:02:49.247 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying podman|docker is present... 2024-04-25T13:02:49.274 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:02:49.274 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying lvm2 is present... 2024-04-25T13:02:49.274 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying time synchronization is in place... 2024-04-25T13:02:49.281 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:02:49.281 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:02:49.287 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:02:49.287 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-25T13:02:49.293 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout enabled 2024-04-25T13:02:49.299 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout active 2024-04-25T13:02:49.300 INFO:teuthology.orchestra.run.smithi072.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:02:49.300 INFO:teuthology.orchestra.run.smithi072.stdout:Repeating the final host check... 2024-04-25T13:02:49.323 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:02:49.324 INFO:teuthology.orchestra.run.smithi072.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T13:02:49.324 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl is present 2024-04-25T13:02:49.324 INFO:teuthology.orchestra.run.smithi072.stdout:lvcreate is present 2024-04-25T13:02:49.330 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:02:49.330 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:02:49.336 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:02:49.336 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-25T13:02:49.342 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout enabled 2024-04-25T13:02:49.348 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout active 2024-04-25T13:02:49.348 INFO:teuthology.orchestra.run.smithi072.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:02:49.348 INFO:teuthology.orchestra.run.smithi072.stdout:Host looks OK 2024-04-25T13:02:49.348 INFO:teuthology.orchestra.run.smithi072.stdout:Cluster fsid: d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:02:49.348 INFO:teuthology.orchestra.run.smithi072.stdout:Acquiring lock 139970177060000 on /run/cephadm/d8ca43ea-0303-11ef-bc93-c7b262605968.lock 2024-04-25T13:02:49.349 INFO:teuthology.orchestra.run.smithi072.stdout:Lock 139970177060000 acquired on /run/cephadm/d8ca43ea-0303-11ef-bc93-c7b262605968.lock 2024-04-25T13:02:49.349 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying IP 172.21.15.72 port 3300 ... 2024-04-25T13:02:49.349 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying IP 172.21.15.72 port 6789 ... 2024-04-25T13:02:49.349 INFO:teuthology.orchestra.run.smithi072.stdout:Base mon IP(s) is [172.21.15.72:3300, 172.21.15.72:6789], mon addrv is [v2:172.21.15.72:3300,v1:172.21.15.72:6789] 2024-04-25T13:02:49.353 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.72 metric 100 2024-04-25T13:02:49.353 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.72 metric 100 2024-04-25T13:02:49.355 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T13:02:49.355 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T13:02:49.355 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T13:02:49.357 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T13:02:49.357 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T13:02:49.357 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:02:49.357 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T13:02:49.358 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout inet6 fe80::4cad:bb47:f4aa:acd5/64 scope link noprefixroute 2024-04-25T13:02:49.358 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:02:49.358 INFO:teuthology.orchestra.run.smithi072.stdout:Mon IP `172.21.15.72` is in CIDR network `172.21.0.0/20` 2024-04-25T13:02:49.358 INFO:teuthology.orchestra.run.smithi072.stdout:Mon IP `172.21.15.72` is in CIDR network `172.21.0.0/20` 2024-04-25T13:02:49.358 INFO:teuthology.orchestra.run.smithi072.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T13:02:49.359 INFO:teuthology.orchestra.run.smithi072.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T13:02:49.360 INFO:teuthology.orchestra.run.smithi072.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:02:49.871 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T13:02:49.872 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:02:49.872 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T13:02:49.872 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T13:02:49.872 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T13:02:49.872 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T13:02:49.872 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T13:02:50.761 INFO:teuthology.orchestra.run.smithi072.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T13:02:50.762 INFO:teuthology.orchestra.run.smithi072.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T13:02:50.762 INFO:teuthology.orchestra.run.smithi072.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T13:02:51.685 INFO:teuthology.orchestra.run.smithi072.stdout:stat: stdout 167 167 2024-04-25T13:02:51.685 INFO:teuthology.orchestra.run.smithi072.stdout:Creating initial keys... 2024-04-25T13:02:52.566 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph-authtool: stdout AQB7VCpmcpCYOxAAr2GiFnV3U1Df9XULzS3pRg== 2024-04-25T13:02:53.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph-authtool: stdout AQB8VCpmyS3gNRAAfwp9t/Iu2DcZCGi8UpkB7A== 2024-04-25T13:02:54.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph-authtool: stdout AQB9VCpmLAqzMhAAY5kGC6Pk++XF2weN7j77oA== 2024-04-25T13:02:54.411 INFO:teuthology.orchestra.run.smithi072.stdout:Creating initial monmap... 2024-04-25T13:02:55.291 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:02:55.291 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T13:02:55.291 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:02:55.291 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:02:55.291 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool for a [v2:172.21.15.72:3300,v1:172.21.15.72:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:02:55.292 INFO:teuthology.orchestra.run.smithi072.stdout:setting min_mon_release = quincy 2024-04-25T13:02:55.292 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: set fsid to d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:02:55.292 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:02:55.292 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:02:55.292 INFO:teuthology.orchestra.run.smithi072.stdout:Creating mon... 2024-04-25T13:02:56.190 INFO:teuthology.orchestra.run.smithi072.stdout:create mon.a on 2024-04-25T13:02:56.387 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T13:02:56.523 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T13:02:56.713 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968.target. 2024-04-25T13:02:56.714 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968.target. 2024-04-25T13:02:56.940 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:02:56.940 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T13:02:57.102 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968.target.wants/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@.service. 2024-04-25T13:02:57.640 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:57 smithi072 ceph-mon[28830]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:02:57.653 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:02:57.653 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-04-25T13:02:57.659 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:02:57.660 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-25T13:02:57.660 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-04-25T13:02:57.660 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:02:57.660 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mon to start... 2024-04-25T13:02:57.660 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mon... 2024-04-25T13:02:57.890 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:57 smithi072 bash[28811]: 5cde398d5849171a348fef87591d13632fcef37e5b4608cb487b8b99d136ec0a 2024-04-25T13:02:57.890 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:57 smithi072 systemd[1]: Started Ceph mon.a for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:02:58.702 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T13:02:58.702 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout id: d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:02:58.702 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T13:02:58.702 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:02:58.702 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout services: 2024-04-25T13:02:58.702 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.571148s) 2024-04-25T13:02:58.703 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T13:02:58.703 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T13:02:58.703 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:02:58.703 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout data: 2024-04-25T13:02:58.703 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T13:02:58.703 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T13:02:58.703 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T13:02:58.703 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T13:02:58.703 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.083+0000 7f2e87857700 1 Processor -- start 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.083+0000 7f2e87857700 1 -- start start 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.083+0000 7f2e87857700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e8010b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.083+0000 7f2e87857700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8007ce70 con 0x7f2e801090c0 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.084+0000 7f2e855f3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e8010b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.084+0000 7f2e855f3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e8010b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45384/0 (socket says 172.21.15.72:45384) 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.084+0000 7f2e855f3700 1 -- 172.21.15.72:0/623066712 learned_addr learned my addr 172.21.15.72:0/623066712 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.084+0000 7f2e855f3700 1 -- 172.21.15.72:0/623066712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8010b9f0 con 0x7f2e801090c0 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.084+0000 7f2e855f3700 1 --2- 172.21.15.72:0/623066712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e8010b4b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0098a0 tx=0x7f2e7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7ef2475eb1fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e77fff700 1 -- 172.21.15.72:0/623066712 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e7c004160 con 0x7f2e801090c0 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e77fff700 1 -- 172.21.15.72:0/623066712 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c004300 con 0x7f2e801090c0 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e77fff700 1 -- 172.21.15.72:0/623066712 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e7c0036a0 con 0x7f2e801090c0 2024-04-25T13:02:58.704 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e87857700 1 -- 172.21.15.72:0/623066712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 msgr2=0x7f2e8010b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e87857700 1 --2- 172.21.15.72:0/623066712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e8010b4b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0098a0 tx=0x7f2e7c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e87857700 1 -- 172.21.15.72:0/623066712 shutdown_connections 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e87857700 1 --2- 172.21.15.72:0/623066712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e8010b4b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e87857700 1 -- 172.21.15.72:0/623066712 >> 172.21.15.72:0/623066712 conn(0x7f2e80102ad0 msgr2=0x7f2e80104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e87857700 1 -- 172.21.15.72:0/623066712 shutdown_connections 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.085+0000 7f2e87857700 1 -- 172.21.15.72:0/623066712 wait complete. 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.086+0000 7f2e87857700 1 Processor -- start 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.086+0000 7f2e87857700 1 -- start start 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.086+0000 7f2e87857700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e80108940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.086+0000 7f2e87857700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8010be60 con 0x7f2e801090c0 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.086+0000 7f2e855f3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e80108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.086+0000 7f2e855f3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e80108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45398/0 (socket says 172.21.15.72:45398) 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.086+0000 7f2e855f3700 1 -- 172.21.15.72:0/196034057 learned_addr learned my addr 172.21.15.72:0/196034057 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:02:58.705 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.086+0000 7f2e855f3700 1 -- 172.21.15.72:0/196034057 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e80108e80 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.087+0000 7f2e855f3700 1 --2- 172.21.15.72:0/196034057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e80108940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.087+0000 7f2e767fc700 1 -- 172.21.15.72:0/196034057 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e7c003db0 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.087+0000 7f2e767fc700 1 -- 172.21.15.72:0/196034057 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c011630 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.087+0000 7f2e87857700 1 -- 172.21.15.72:0/196034057 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e80107070 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.087+0000 7f2e767fc700 1 -- 172.21.15.72:0/196034057 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e7c011950 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.087+0000 7f2e87857700 1 -- 172.21.15.72:0/196034057 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e801075b0 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.087+0000 7f2e767fc700 1 -- 172.21.15.72:0/196034057 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e7c011af0 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.087+0000 7f2e767fc700 1 -- 172.21.15.72:0/196034057 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2e7c01c450 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.088+0000 7f2e87857700 1 -- 172.21.15.72:0/196034057 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e800640a0 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.090+0000 7f2e767fc700 1 -- 172.21.15.72:0/196034057 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2e7c032020 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.159+0000 7f2e87857700 1 -- 172.21.15.72:0/196034057 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f2e80195660 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.159+0000 7f2e767fc700 1 -- 172.21.15.72:0/196034057 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f2e7c004260 con 0x7f2e801090c0 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.160+0000 7f2e87857700 1 -- 172.21.15.72:0/196034057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 msgr2=0x7f2e80108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.706 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.160+0000 7f2e87857700 1 --2- 172.21.15.72:0/196034057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e80108940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c004750 comp rx=0 tx=0).stop 2024-04-25T13:02:58.707 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.160+0000 7f2e87857700 1 -- 172.21.15.72:0/196034057 shutdown_connections 2024-04-25T13:02:58.707 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.160+0000 7f2e87857700 1 --2- 172.21.15.72:0/196034057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e801090c0 0x7f2e80108940 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.707 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.160+0000 7f2e87857700 1 -- 172.21.15.72:0/196034057 >> 172.21.15.72:0/196034057 conn(0x7f2e80102ad0 msgr2=0x7f2e80104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:58.707 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.160+0000 7f2e87857700 1 -- 172.21.15.72:0/196034057 shutdown_connections 2024-04-25T13:02:58.707 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:58.160+0000 7f2e87857700 1 -- 172.21.15.72:0/196034057 wait complete. 2024-04-25T13:02:58.707 INFO:teuthology.orchestra.run.smithi072.stdout:mon is available 2024-04-25T13:02:58.707 INFO:teuthology.orchestra.run.smithi072.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T13:02:58.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:02:58.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: monmap epoch 1 2024-04-25T13:02:58.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:02:58.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: last_changed 2024-04-25T13:02:54.770882+0000 2024-04-25T13:02:58.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: created 2024-04-25T13:02:54.770882+0000 2024-04-25T13:02:58.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: min_mon_release 19 (squid) 2024-04-25T13:02:58.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: election_strategy: 1 2024-04-25T13:02:58.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-25T13:02:58.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: fsmap 2024-04-25T13:02:58.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:02:58.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: mgrmap e1: no daemons active 2024-04-25T13:02:58.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:58 smithi072 ceph-mon[28830]: from='client.? 172.21.15.72:0/196034057' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T13:02:59.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:59 smithi072 ceph-mon[28830]: from='client.? 172.21.15.72:0/4001564770' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:02:59.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:02:59 smithi072 ceph-mon[28830]: from='client.? 172.21.15.72:0/4001564770' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T13:02:59.817 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:02:59.817 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:02:59.817 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout fsid = d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:02:59.817 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:02:59.817 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.72:3300,v1:172.21.15.72:6789] 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T13:02:59.818 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.114+0000 7fe378cb5700 1 Processor -- start 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.114+0000 7fe378cb5700 1 -- start start 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.114+0000 7fe378cb5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe374107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.114+0000 7fe378cb5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe374107f00 con 0x7fe374107510 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.114+0000 7fe37259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe374107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.114+0000 7fe37259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe374107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45414/0 (socket says 172.21.15.72:45414) 2024-04-25T13:02:59.819 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.114+0000 7fe37259c700 1 -- 172.21.15.72:0/1230254240 learned_addr learned my addr 172.21.15.72:0/1230254240 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe37259c700 1 -- 172.21.15.72:0/1230254240 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe374108710 con 0x7fe374107510 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe37259c700 1 --2- 172.21.15.72:0/1230254240 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe374107930 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe3640098a0 tx=0x7fe364009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc71462d8faaf3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe37159a700 1 -- 172.21.15.72:0/1230254240 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe364004160 con 0x7fe374107510 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe37159a700 1 -- 172.21.15.72:0/1230254240 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe364004300 con 0x7fe374107510 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe37159a700 1 -- 172.21.15.72:0/1230254240 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3640036a0 con 0x7fe374107510 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe378cb5700 1 -- 172.21.15.72:0/1230254240 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 msgr2=0x7fe374107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe378cb5700 1 --2- 172.21.15.72:0/1230254240 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe374107930 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe3640098a0 tx=0x7fe364009bb0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe378cb5700 1 -- 172.21.15.72:0/1230254240 shutdown_connections 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe378cb5700 1 --2- 172.21.15.72:0/1230254240 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe374107930 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.115+0000 7fe378cb5700 1 -- 172.21.15.72:0/1230254240 >> 172.21.15.72:0/1230254240 conn(0x7fe374102ad0 msgr2=0x7fe374104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.116+0000 7fe378cb5700 1 -- 172.21.15.72:0/1230254240 shutdown_connections 2024-04-25T13:02:59.820 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.116+0000 7fe378cb5700 1 -- 172.21.15.72:0/1230254240 wait complete. 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.116+0000 7fe378cb5700 1 Processor -- start 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.116+0000 7fe378cb5700 1 -- start start 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.116+0000 7fe378cb5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe37407ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.116+0000 7fe378cb5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe374108c40 con 0x7fe374107510 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe37259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe37407ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe37259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe37407ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45416/0 (socket says 172.21.15.72:45416) 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe37259c700 1 -- 172.21.15.72:0/4001564770 learned_addr learned my addr 172.21.15.72:0/4001564770 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe37259c700 1 -- 172.21.15.72:0/4001564770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37419e9a0 con 0x7fe374107510 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe37259c700 1 --2- 172.21.15.72:0/4001564770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe37407ca50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe364000c00 tx=0x7fe364004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe3637fe700 1 -- 172.21.15.72:0/4001564770 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe364003db0 con 0x7fe374107510 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe3637fe700 1 -- 172.21.15.72:0/4001564770 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe364011630 con 0x7fe374107510 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe378cb5700 1 -- 172.21.15.72:0/4001564770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37419eb70 con 0x7fe374107510 2024-04-25T13:02:59.821 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe3637fe700 1 -- 172.21.15.72:0/4001564770 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe364011950 con 0x7fe374107510 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe378cb5700 1 -- 172.21.15.72:0/4001564770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37419ef10 con 0x7fe374107510 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe3637fe700 1 -- 172.21.15.72:0/4001564770 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe364011af0 con 0x7fe374107510 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.117+0000 7fe3637fe700 1 -- 172.21.15.72:0/4001564770 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe36401c450 con 0x7fe374107510 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.118+0000 7fe378cb5700 1 -- 172.21.15.72:0/4001564770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3740640a0 con 0x7fe374107510 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.120+0000 7fe3637fe700 1 -- 172.21.15.72:0/4001564770 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe364032020 con 0x7fe374107510 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.187+0000 7fe378cb5700 1 -- 172.21.15.72:0/4001564770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe3740008d0 con 0x7fe374107510 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.190+0000 7fe3637fe700 1 -- 172.21.15.72:0/4001564770 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7fe364004260 con 0x7fe374107510 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.191+0000 7fe3637fe700 1 -- 172.21.15.72:0/4001564770 <== mon.0 v2:172.21.15.72:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe36401c6a0 con 0x7fe374107510 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.191+0000 7fe378cb5700 1 -- 172.21.15.72:0/4001564770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 msgr2=0x7fe37407ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.191+0000 7fe378cb5700 1 --2- 172.21.15.72:0/4001564770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe37407ca50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe364000c00 tx=0x7fe364004750 comp rx=0 tx=0).stop 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.191+0000 7fe378cb5700 1 -- 172.21.15.72:0/4001564770 shutdown_connections 2024-04-25T13:02:59.822 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.192+0000 7fe378cb5700 1 --2- 172.21.15.72:0/4001564770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe374107510 0x7fe37407ca50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.823 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.192+0000 7fe378cb5700 1 -- 172.21.15.72:0/4001564770 >> 172.21.15.72:0/4001564770 conn(0x7fe374102ad0 msgr2=0x7fe374191870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:59.823 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.192+0000 7fe378cb5700 1 -- 172.21.15.72:0/4001564770 shutdown_connections 2024-04-25T13:02:59.823 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:02:59.192+0000 7fe378cb5700 1 -- 172.21.15.72:0/4001564770 wait complete. 2024-04-25T13:02:59.823 INFO:teuthology.orchestra.run.smithi072.stdout:Generating new minimal ceph.conf... 2024-04-25T13:03:00.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.244+0000 7ff650994700 1 Processor -- start 2024-04-25T13:03:00.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.244+0000 7ff650994700 1 -- start start 2024-04-25T13:03:00.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.244+0000 7ff650994700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c106d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.244+0000 7ff650994700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c107300 con 0x7ff64c106910 2024-04-25T13:03:00.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.244+0000 7ff64a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c106d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.245+0000 7ff64a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c106d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45432/0 (socket says 172.21.15.72:45432) 2024-04-25T13:03:00.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.245+0000 7ff64a59c700 1 -- 172.21.15.72:0/1902211922 learned_addr learned my addr 172.21.15.72:0/1902211922 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.245+0000 7ff64a59c700 1 -- 172.21.15.72:0/1902211922 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c107480 con 0x7ff64c106910 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.245+0000 7ff64a59c700 1 --2- 172.21.15.72:0/1902211922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c106d30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0098a0 tx=0x7ff63c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16a6e40bf31e4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.245+0000 7ff649d9b700 1 -- 172.21.15.72:0/1902211922 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff63c004160 con 0x7ff64c106910 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.245+0000 7ff649d9b700 1 -- 172.21.15.72:0/1902211922 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff63c004300 con 0x7ff64c106910 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.245+0000 7ff649d9b700 1 -- 172.21.15.72:0/1902211922 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff63c011af0 con 0x7ff64c106910 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.246+0000 7ff650994700 1 -- 172.21.15.72:0/1902211922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 msgr2=0x7ff64c106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.246+0000 7ff650994700 1 --2- 172.21.15.72:0/1902211922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c106d30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0098a0 tx=0x7ff63c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.246+0000 7ff650994700 1 -- 172.21.15.72:0/1902211922 shutdown_connections 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.246+0000 7ff650994700 1 --2- 172.21.15.72:0/1902211922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c106d30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.246+0000 7ff650994700 1 -- 172.21.15.72:0/1902211922 >> 172.21.15.72:0/1902211922 conn(0x7ff64c1022b0 msgr2=0x7ff64c1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.246+0000 7ff650994700 1 -- 172.21.15.72:0/1902211922 shutdown_connections 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.246+0000 7ff650994700 1 -- 172.21.15.72:0/1902211922 wait complete. 2024-04-25T13:03:00.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.246+0000 7ff650994700 1 Processor -- start 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff650994700 1 -- start start 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff650994700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff650994700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10a780 con 0x7ff64c106910 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff64a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff64a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c19c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45434/0 (socket says 172.21.15.72:45434) 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff64a59c700 1 -- 172.21.15.72:0/3495177607 learned_addr learned my addr 172.21.15.72:0/3495177607 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff64a59c700 1 -- 172.21.15.72:0/3495177607 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c19cad0 con 0x7ff64c106910 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff64a59c700 1 --2- 172.21.15.72:0/3495177607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c19c590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0117b0 tx=0x7ff63c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff643fff700 1 -- 172.21.15.72:0/3495177607 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff63c003af0 con 0x7ff64c106910 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff650994700 1 -- 172.21.15.72:0/3495177607 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c19cd60 con 0x7ff64c106910 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff643fff700 1 -- 172.21.15.72:0/3495177607 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff63c020070 con 0x7ff64c106910 2024-04-25T13:03:00.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff643fff700 1 -- 172.21.15.72:0/3495177607 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff63c019630 con 0x7ff64c106910 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.247+0000 7ff650994700 1 -- 172.21.15.72:0/3495177607 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c19fa50 con 0x7ff64c106910 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.248+0000 7ff643fff700 1 -- 172.21.15.72:0/3495177607 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff63c019c80 con 0x7ff64c106910 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.248+0000 7ff643fff700 1 -- 172.21.15.72:0/3495177607 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff63c00b720 con 0x7ff64c106910 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.249+0000 7ff650994700 1 -- 172.21.15.72:0/3495177607 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64c0640a0 con 0x7ff64c106910 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.251+0000 7ff643fff700 1 -- 172.21.15.72:0/3495177607 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff63c0042a0 con 0x7ff64c106910 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.318+0000 7ff650994700 1 -- 172.21.15.72:0/3495177607 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff64c19ff90 con 0x7ff64c106910 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.319+0000 7ff643fff700 1 -- 172.21.15.72:0/3495177607 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff63c01a9c0 con 0x7ff64c106910 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.319+0000 7ff650994700 1 -- 172.21.15.72:0/3495177607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 msgr2=0x7ff64c19c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.319+0000 7ff650994700 1 --2- 172.21.15.72:0/3495177607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c19c590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0117b0 tx=0x7ff63c011b70 comp rx=0 tx=0).stop 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.319+0000 7ff650994700 1 -- 172.21.15.72:0/3495177607 shutdown_connections 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.319+0000 7ff650994700 1 --2- 172.21.15.72:0/3495177607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff64c106910 0x7ff64c19c590 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.319+0000 7ff650994700 1 -- 172.21.15.72:0/3495177607 >> 172.21.15.72:0/3495177607 conn(0x7ff64c1022b0 msgr2=0x7ff64c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:00.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.320+0000 7ff650994700 1 -- 172.21.15.72:0/3495177607 shutdown_connections 2024-04-25T13:03:00.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:00.320+0000 7ff650994700 1 -- 172.21.15.72:0/3495177607 wait complete. 2024-04-25T13:03:00.850 INFO:teuthology.orchestra.run.smithi072.stdout:Restarting the monitor... 2024-04-25T13:03:00.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:00 smithi072 ceph-mon[28830]: from='client.? 172.21.15.72:0/3495177607' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:03:01.114 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:00 smithi072 systemd[1]: Stopping Ceph mon.a for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:03:01.114 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a[28826]: 2024-04-25T13:03:01.094+0000 7f5a7a840700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:03:01.114 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a[28826]: 2024-04-25T13:03:01.094+0000 7f5a7a840700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T13:03:01.387 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 podman[29074]: 2024-04-25 13:03:01.116063612 +0000 UTC m=+0.111014863 container died 5cde398d5849171a348fef87591d13632fcef37e5b4608cb487b8b99d136ec0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:03:01.723 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 podman[29074]: 2024-04-25 13:03:01.42170705 +0000 UTC m=+0.416658303 container cleanup 5cde398d5849171a348fef87591d13632fcef37e5b4608cb487b8b99d136ec0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:03:01.723 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 bash[29074]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:03:01.723 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 podman[29088]: 2024-04-25 13:03:01.616695967 +0000 UTC m=+0.504025640 container remove 5cde398d5849171a348fef87591d13632fcef37e5b4608cb487b8b99d136ec0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5) 2024-04-25T13:03:02.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 systemd[1]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T13:03:02.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 systemd[1]: Stopped Ceph mon.a for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:03:02.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 systemd[1]: Starting Ceph mon.a for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:03:02.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 podman[29180]: 2024-04-25 13:03:01.960201526 +0000 UTC m=+0.113940851 container create f54ab9bf98cbd2517159d9806b4f360e88cf32bd041be04d0ff8b0fdd263ba46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T13:03:02.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:01 smithi072 podman[29180]: 2024-04-25 13:03:01.866610394 +0000 UTC m=+0.020349718 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:03:02.209 INFO:teuthology.orchestra.run.smithi072.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T13:03:02.334 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 podman[29180]: 2024-04-25 13:03:02.130818766 +0000 UTC m=+0.284558091 container init f54ab9bf98cbd2517159d9806b4f360e88cf32bd041be04d0ff8b0fdd263ba46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T13:03:02.334 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 podman[29180]: 2024-04-25 13:03:02.133678284 +0000 UTC m=+0.287417611 container start f54ab9bf98cbd2517159d9806b4f360e88cf32bd041be04d0ff8b0fdd263ba46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T13:03:02.334 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T13:03:02.334 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: pidfile_write: ignore empty --pid-file 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: load: jerasure load: lrc 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: RocksDB version: 7.9.2 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Git sha 0 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: DB SUMMARY 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: DB Session ID: TZLASVK3A8Q33T7PZL6C 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: CURRENT file: CURRENT 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106028 ; 2024-04-25T13:03:02.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.error_if_exists: 0 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.create_if_missing: 0 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.paranoid_checks: 1 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.env: 0x55c376e26cc0 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.info_log: 0x55c378c8fbe0 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.statistics: (nil) 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.use_fsync: 0 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_log_file_size: 0 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T13:03:02.336 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.allow_fallocate: 1 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.use_direct_reads: 0 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.db_log_dir: 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.wal_dir: 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T13:03:02.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.write_buffer_manager: 0x55c378c230e0 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T13:03:02.338 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T13:03:02.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T13:03:02.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.unordered_write: 0 2024-04-25T13:03:02.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T13:03:02.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T13:03:02.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T13:03:02.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T13:03:02.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.row_cache: None 2024-04-25T13:03:02.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.wal_filter: None 2024-04-25T13:03:02.340 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T13:03:02.340 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T13:03:02.340 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.two_write_queues: 0 2024-04-25T13:03:02.340 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.wal_compression: 0 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.atomic_flush: 0 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.log_readahead_size: 0 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T13:03:02.341 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_background_jobs: 2 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_background_compactions: -1 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_subcompactions: 1 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_open_files: -1 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T13:03:02.342 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_background_flushes: -1 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Compression algorithms supported: 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: kZSTD supported: 0 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: kXpressCompression supported: 0 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: kZlibCompression supported: 1 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: kSnappyCompression supported: 1 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: kLZ4Compression supported: 1 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: kBZip2Compression supported: 0 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T13:03:02.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.merge_operator: 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_filter: None 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c378c8fdc0) 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout: cache_index_and_filter_blocks: 1 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout: pin_top_level_index_and_filter: 1 2024-04-25T13:03:02.344 INFO:journalctl@ceph.mon.a.smithi072.stdout: index_type: 0 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: data_block_index_type: 0 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: index_shortening: 1 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: checksum: 4 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: no_block_cache: 0 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache: 0x55c378c551f0 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache_name: BinnedLRUCache 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache_options: 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: capacity : 536870912 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: num_shard_bits : 4 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: strict_capacity_limit : 0 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: high_pri_pool_ratio: 0.000 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache_compressed: (nil) 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: persistent_cache: (nil) 2024-04-25T13:03:02.345 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_size: 4096 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_size_deviation: 10 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_restart_interval: 16 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: index_block_restart_interval: 1 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: metadata_block_size: 4096 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: partition_filters: 0 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: use_delta_encoding: 1 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: filter_policy: bloomfilter 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: whole_key_filtering: 1 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: verify_compression: 0 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: read_amp_bytes_per_bit: 0 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: format_version: 5 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: enable_index_compression: 1 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_align: 0 2024-04-25T13:03:02.346 INFO:journalctl@ceph.mon.a.smithi072.stdout: max_auto_readahead_size: 262144 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout: prepopulate_block_cache: 0 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout: initial_auto_readahead_size: 8192 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression: NoCompression 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.num_levels: 7 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T13:03:02.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:03:02.348 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T13:03:02.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T13:03:02.350 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.inplace_update_support: 0 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.bloom_locality: 0 2024-04-25T13:03:02.351 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.max_successive_merges: 0 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.ttl: 2592000 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.enable_blob_files: false 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.min_blob_size: 0 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T13:03:02.352 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 53036aa9-3ae7-4954-a5f3-6de991b61d28 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050182162162, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050182163124, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98969, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91148, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714050182, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "53036aa9-3ae7-4954-a5f3-6de991b61d28", "db_session_id": "TZLASVK3A8Q33T7PZL6C", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050182163205, "job": 1, "event": "recovery_finished"} 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T13:03:02.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c378d6a000 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: DB pointer 0x55c378d54000 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** DB Stats ** 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-25T13:03:02.354 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** Compaction Stats [default] ** 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: L0 2/0 100.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 162.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: Sum 2/0 100.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 162.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 162.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** Compaction Stats [default] ** 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 162.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-25T13:03:02.355 INFO:journalctl@ceph.mon.a.smithi072.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative compaction: 0.00 GB write, 32.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval compaction: 0.00 GB write, 32.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: Block cache BinnedLRUCache@0x55c378c551f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: starting mon.a rank 0 at public addrs [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] at bind addrs [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:03:02.356 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mon.a@-1(???) e1 preinit fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mon.a@-1(???).mds e1 new map 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mon.a@-1(???).mds e1 print_map 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout: e1 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout: legacy client fscid: -1 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout: No filesystems configured 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T13:03:02.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 bash[29180]: f54ab9bf98cbd2517159d9806b4f360e88cf32bd041be04d0ff8b0fdd263ba46 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 systemd[1]: Started Ceph mon.a for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: monmap epoch 1 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: last_changed 2024-04-25T13:02:54.770882+0000 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: created 2024-04-25T13:02:54.770882+0000 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: min_mon_release 19 (squid) 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: election_strategy: 1 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: fsmap 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:03:02.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:02 smithi072 ceph-mon[29199]: mgrmap e1: no daemons active 2024-04-25T13:03:03.305 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.680+0000 7f52abca0700 1 Processor -- start 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.680+0000 7f52abca0700 1 -- start start 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.680+0000 7f52abca0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a4108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.680+0000 7f52abca0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f52a4109060 con 0x7f52a4108670 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.680+0000 7f52a9a3c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a4108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.680+0000 7f52a9a3c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a4108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45450/0 (socket says 172.21.15.72:45450) 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.680+0000 7f52a9a3c700 1 -- 172.21.15.72:0/2458358476 learned_addr learned my addr 172.21.15.72:0/2458358476 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.681+0000 7f52a9a3c700 1 -- 172.21.15.72:0/2458358476 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a4109820 con 0x7f52a4108670 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.681+0000 7f52a9a3c700 1 --2- 172.21.15.72:0/2458358476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a4108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f52a00098a0 tx=0x7f52a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630887cdfdf75960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.681+0000 7f52a8a3a700 1 -- 172.21.15.72:0/2458358476 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52a0004160 con 0x7f52a4108670 2024-04-25T13:03:03.306 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.681+0000 7f52a8a3a700 1 -- 172.21.15.72:0/2458358476 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f52a0004300 con 0x7f52a4108670 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.681+0000 7f52a8a3a700 1 -- 172.21.15.72:0/2458358476 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52a0011af0 con 0x7f52a4108670 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.682+0000 7f52abca0700 1 -- 172.21.15.72:0/2458358476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 msgr2=0x7f52a4108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.682+0000 7f52abca0700 1 --2- 172.21.15.72:0/2458358476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a4108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f52a00098a0 tx=0x7f52a0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.682+0000 7f52abca0700 1 -- 172.21.15.72:0/2458358476 shutdown_connections 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.682+0000 7f52abca0700 1 --2- 172.21.15.72:0/2458358476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a4108a90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.682+0000 7f52abca0700 1 -- 172.21.15.72:0/2458358476 >> 172.21.15.72:0/2458358476 conn(0x7f52a4103c30 msgr2=0x7f52a4106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.682+0000 7f52abca0700 1 -- 172.21.15.72:0/2458358476 shutdown_connections 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.682+0000 7f52abca0700 1 -- 172.21.15.72:0/2458358476 wait complete. 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.682+0000 7f52abca0700 1 Processor -- start 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.682+0000 7f52abca0700 1 -- start start 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f52abca0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f52abca0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f52a410ca50 con 0x7f52a4108670 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f52a9a3c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:03.307 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f52a9a3c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45454/0 (socket says 172.21.15.72:45454) 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f52a9a3c700 1 -- 172.21.15.72:0/2280745521 learned_addr learned my addr 172.21.15.72:0/2280745521 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f52a9a3c700 1 -- 172.21.15.72:0/2280745521 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a419ca90 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f52a9a3c700 1 --2- 172.21.15.72:0/2280745521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a419c550 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009fb0 tx=0x7f52a0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f529affd700 1 -- 172.21.15.72:0/2280745521 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52a00036a0 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f529affd700 1 -- 172.21.15.72:0/2280745521 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f52a001a430 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f529affd700 1 -- 172.21.15.72:0/2280745521 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52a0019550 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f52abca0700 1 -- 172.21.15.72:0/2280745521 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a419cd20 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f52abca0700 1 -- 172.21.15.72:0/2280745521 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a419fa10 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.683+0000 7f529affd700 1 -- 172.21.15.72:0/2280745521 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f52a0019be0 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.684+0000 7f529affd700 1 -- 172.21.15.72:0/2280745521 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f52a0024730 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.684+0000 7f52abca0700 1 -- 172.21.15.72:0/2280745521 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a410cbd0 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.687+0000 7f529affd700 1 -- 172.21.15.72:0/2280745521 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f52a0043e50 con 0x7f52a4108670 2024-04-25T13:03:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.755+0000 7f52abca0700 1 -- 172.21.15.72:0/2280745521 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f52a419fc50 con 0x7f52a4108670 2024-04-25T13:03:03.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.757+0000 7f529affd700 1 -- 172.21.15.72:0/2280745521 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f52a0032020 con 0x7f52a4108670 2024-04-25T13:03:03.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.757+0000 7f52abca0700 1 -- 172.21.15.72:0/2280745521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 msgr2=0x7f52a419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:03.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.757+0000 7f52abca0700 1 --2- 172.21.15.72:0/2280745521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a419c550 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009fb0 tx=0x7f52a0011940 comp rx=0 tx=0).stop 2024-04-25T13:03:03.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.757+0000 7f52abca0700 1 -- 172.21.15.72:0/2280745521 shutdown_connections 2024-04-25T13:03:03.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.757+0000 7f52abca0700 1 --2- 172.21.15.72:0/2280745521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52a4108670 0x7f52a419c550 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.757+0000 7f52abca0700 1 -- 172.21.15.72:0/2280745521 >> 172.21.15.72:0/2280745521 conn(0x7f52a4103c30 msgr2=0x7f52a4104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:03.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.757+0000 7f52abca0700 1 -- 172.21.15.72:0/2280745521 shutdown_connections 2024-04-25T13:03:03.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:02.758+0000 7f52abca0700 1 -- 172.21.15.72:0/2280745521 wait complete. 2024-04-25T13:03:03.309 INFO:teuthology.orchestra.run.smithi072.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T13:03:03.332 INFO:teuthology.orchestra.run.smithi072.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:03.332 INFO:teuthology.orchestra.run.smithi072.stdout:Creating mgr... 2024-04-25T13:03:03.333 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T13:03:03.333 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T13:03:03.570 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:03:03.570 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T13:03:03.713 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968.target.wants/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@.service. 2024-04-25T13:03:04.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:03 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2280745521' entity='client.admin' 2024-04-25T13:03:04.022 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:03 smithi072 podman[29417]: 2024-04-25 13:03:03.851609142 +0000 UTC m=+0.018003717 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:03:04.234 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:03:04.234 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-04-25T13:03:04.241 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:03:04.241 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-25T13:03:04.241 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-04-25T13:03:04.242 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:03:04.250 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:03:04.250 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-04-25T13:03:04.257 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:03:04.257 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-25T13:03:04.257 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-04-25T13:03:04.257 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T13:03:04.258 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr to start... 2024-04-25T13:03:04.258 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr... 2024-04-25T13:03:04.288 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:04 smithi072 podman[29417]: 2024-04-25 13:03:04.129577547 +0000 UTC m=+0.295972095 container init fafea5b7b7d7652c4dbd1f2c214bd60f6fa5dce10361d6ef8f121bac15ade058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T13:03:04.288 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:04 smithi072 podman[29417]: 2024-04-25 13:03:04.132089684 +0000 UTC m=+0.298484232 container start fafea5b7b7d7652c4dbd1f2c214bd60f6fa5dce10361d6ef8f121bac15ade058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T13:03:04.288 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:04 smithi072 bash[29417]: fafea5b7b7d7652c4dbd1f2c214bd60f6fa5dce10361d6ef8f121bac15ade058 2024-04-25T13:03:04.288 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:04 smithi072 systemd[1]: Started Ceph mgr.x for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:03:04.579 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:04.289+0000 7f6f06110200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:03:04.579 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:04.477+0000 7f6f06110200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:04.579 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:04.580+0000 7f6f06110200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:03:05.239 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:04 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1261184221' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:03:05.239 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:04.902+0000 7f6f06110200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:03:05.239 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:04.987+0000 7f6f06110200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:03:05.514 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:03:05.514 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:03:05.514 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:03:05.514 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:03:05.514 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:03:05.514 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:03:05.515 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:03:05.516 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:05.516 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:03:05.516 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:05.516 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:03:05.516 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:03:05.516 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:03:05.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:03:05.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:02:57.590496+0000", 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.702+0000 7fb72e672700 1 Processor -- start 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.702+0000 7fb72e672700 1 -- start start 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.702+0000 7fb72e672700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb7281070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:05.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.702+0000 7fb72e672700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb7281076a0 con 0x7fb728106cb0 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.703+0000 7fb727fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb7281070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.703+0000 7fb727fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb7281070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45480/0 (socket says 172.21.15.72:45480) 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.703+0000 7fb727fff700 1 -- 172.21.15.72:0/4182350779 learned_addr learned my addr 172.21.15.72:0/4182350779 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.703+0000 7fb727fff700 1 -- 172.21.15.72:0/4182350779 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb728107eb0 con 0x7fb728106cb0 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.703+0000 7fb727fff700 1 --2- 172.21.15.72:0/4182350779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb7281070d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0098a0 tx=0x7fb71c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3925538f9e8d958b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.703+0000 7fb726ffd700 1 -- 172.21.15.72:0/4182350779 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb71c004160 con 0x7fb728106cb0 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.703+0000 7fb726ffd700 1 -- 172.21.15.72:0/4182350779 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb71c004300 con 0x7fb728106cb0 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.703+0000 7fb726ffd700 1 -- 172.21.15.72:0/4182350779 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb71c011af0 con 0x7fb728106cb0 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.704+0000 7fb72e672700 1 -- 172.21.15.72:0/4182350779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 msgr2=0x7fb7281070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.704+0000 7fb72e672700 1 --2- 172.21.15.72:0/4182350779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb7281070d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0098a0 tx=0x7fb71c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.704+0000 7fb72e672700 1 -- 172.21.15.72:0/4182350779 shutdown_connections 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.704+0000 7fb72e672700 1 --2- 172.21.15.72:0/4182350779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb7281070d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.704+0000 7fb72e672700 1 -- 172.21.15.72:0/4182350779 >> 172.21.15.72:0/4182350779 conn(0x7fb728102270 msgr2=0x7fb728104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.704+0000 7fb72e672700 1 -- 172.21.15.72:0/4182350779 shutdown_connections 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.704+0000 7fb72e672700 1 -- 172.21.15.72:0/4182350779 wait complete. 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.705+0000 7fb72e672700 1 Processor -- start 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.705+0000 7fb72e672700 1 -- start start 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.705+0000 7fb72e672700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb72818f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.705+0000 7fb72e672700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb728108570 con 0x7fb728106cb0 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.705+0000 7fb727fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb72818f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.705+0000 7fb727fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb72818f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45496/0 (socket says 172.21.15.72:45496) 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.705+0000 7fb727fff700 1 -- 172.21.15.72:0/1261184221 learned_addr learned my addr 172.21.15.72:0/1261184221 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.705+0000 7fb727fff700 1 -- 172.21.15.72:0/1261184221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72818fdb0 con 0x7fb728106cb0 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.705+0000 7fb727fff700 1 --2- 172.21.15.72:0/1261184221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb72818f870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0117b0 tx=0x7fb71c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.706+0000 7fb7257fa700 1 -- 172.21.15.72:0/1261184221 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb71c003af0 con 0x7fb728106cb0 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.706+0000 7fb7257fa700 1 -- 172.21.15.72:0/1261184221 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb71c020070 con 0x7fb728106cb0 2024-04-25T13:03:05.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.706+0000 7fb72e672700 1 -- 172.21.15.72:0/1261184221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72818dfa0 con 0x7fb728106cb0 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.706+0000 7fb7257fa700 1 -- 172.21.15.72:0/1261184221 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb71c019630 con 0x7fb728106cb0 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.706+0000 7fb72e672700 1 -- 172.21.15.72:0/1261184221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72818e4e0 con 0x7fb728106cb0 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.706+0000 7fb7257fa700 1 -- 172.21.15.72:0/1261184221 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb71c0198f0 con 0x7fb728106cb0 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.706+0000 7fb7257fa700 1 -- 172.21.15.72:0/1261184221 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb71c00b3e0 con 0x7fb728106cb0 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.707+0000 7fb72e672700 1 -- 172.21.15.72:0/1261184221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7280640a0 con 0x7fb728106cb0 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.709+0000 7fb7257fa700 1 -- 172.21.15.72:0/1261184221 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb71c0042a0 con 0x7fb728106cb0 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.779+0000 7fb72e672700 1 -- 172.21.15.72:0/1261184221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb728195dd0 con 0x7fb728106cb0 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.779+0000 7fb7257fa700 1 -- 172.21.15.72:0/1261184221 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb71c025e20 con 0x7fb728106cb0 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.781+0000 7fb72e672700 1 -- 172.21.15.72:0/1261184221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 msgr2=0x7fb72818f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.781+0000 7fb72e672700 1 --2- 172.21.15.72:0/1261184221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb72818f870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0117b0 tx=0x7fb71c011b70 comp rx=0 tx=0).stop 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.781+0000 7fb72e672700 1 -- 172.21.15.72:0/1261184221 shutdown_connections 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.781+0000 7fb72e672700 1 --2- 172.21.15.72:0/1261184221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb728106cb0 0x7fb72818f870 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.781+0000 7fb72e672700 1 -- 172.21.15.72:0/1261184221 >> 172.21.15.72:0/1261184221 conn(0x7fb728102270 msgr2=0x7fb7281931b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:05.529 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.781+0000 7fb72e672700 1 -- 172.21.15.72:0/1261184221 shutdown_connections 2024-04-25T13:03:05.530 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:04.781+0000 7fb72e672700 1 -- 172.21.15.72:0/1261184221 wait complete. 2024-04-25T13:03:05.530 INFO:teuthology.orchestra.run.smithi072.stdout:mgr not available, waiting (1/15)... 2024-04-25T13:03:06.418 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:06.164+0000 7f6f06110200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:03:06.418 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:06.328+0000 7f6f06110200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:06.772 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:06.505+0000 7f6f06110200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:03:07.075 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:06.788+0000 7f6f06110200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:03:07.075 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:06.878+0000 7f6f06110200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:03:07.075 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:07.076+0000 7f6f06110200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:03:07.522 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:07.264+0000 7f6f06110200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:07.900 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:07.865+0000 7f6f06110200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:03:08.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:08 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1039421231' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:03:08.574 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:08 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:08.295+0000 7f6f06110200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:03:08.574 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:08 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:08.393+0000 7f6f06110200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:03:08.579 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:03:08.579 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:03:08.579 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:03:08.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:03:08.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:03:08.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:03:08.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:03:08.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:08.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:08.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:03:08.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:02:57.590496+0000", 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.913+0000 7ff5064cc700 1 Processor -- start 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.913+0000 7ff5064cc700 1 -- start start 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.913+0000 7ff5064cc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50010b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.913+0000 7ff5064cc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff50007ce90 con 0x7ff500109080 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.913+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50010b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.913+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50010b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35752/0 (socket says 172.21.15.72:35752) 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.913+0000 7ff4fffff700 1 -- 172.21.15.72:0/3219158180 learned_addr learned my addr 172.21.15.72:0/3219158180 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:08.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.914+0000 7ff4fffff700 1 -- 172.21.15.72:0/3219158180 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50010b9b0 con 0x7ff500109080 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.914+0000 7ff4fffff700 1 --2- 172.21.15.72:0/3219158180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50010b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40098a0 tx=0x7ff4f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47f66985e577e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.914+0000 7ff4feffd700 1 -- 172.21.15.72:0/3219158180 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4f4004160 con 0x7ff500109080 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.914+0000 7ff4feffd700 1 -- 172.21.15.72:0/3219158180 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff4f4004300 con 0x7ff500109080 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.914+0000 7ff4feffd700 1 -- 172.21.15.72:0/3219158180 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4f4011af0 con 0x7ff500109080 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.915+0000 7ff5064cc700 1 -- 172.21.15.72:0/3219158180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 msgr2=0x7ff50010b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.915+0000 7ff5064cc700 1 --2- 172.21.15.72:0/3219158180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50010b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40098a0 tx=0x7ff4f4009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.915+0000 7ff5064cc700 1 -- 172.21.15.72:0/3219158180 shutdown_connections 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.915+0000 7ff5064cc700 1 --2- 172.21.15.72:0/3219158180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50010b470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.915+0000 7ff5064cc700 1 -- 172.21.15.72:0/3219158180 >> 172.21.15.72:0/3219158180 conn(0x7ff500102af0 msgr2=0x7ff500104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.915+0000 7ff5064cc700 1 -- 172.21.15.72:0/3219158180 shutdown_connections 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.915+0000 7ff5064cc700 1 -- 172.21.15.72:0/3219158180 wait complete. 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.915+0000 7ff5064cc700 1 Processor -- start 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.915+0000 7ff5064cc700 1 -- start start 2024-04-25T13:03:08.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.916+0000 7ff5064cc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.916+0000 7ff5064cc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff50010c8a0 con 0x7ff500109080 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.916+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.916+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35756/0 (socket says 172.21.15.72:35756) 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.916+0000 7ff4fffff700 1 -- 172.21.15.72:0/1039421231 learned_addr learned my addr 172.21.15.72:0/1039421231 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.916+0000 7ff4fffff700 1 -- 172.21.15.72:0/1039421231 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50019ca10 con 0x7ff500109080 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.916+0000 7ff4fffff700 1 --2- 172.21.15.72:0/1039421231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50019c4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4011ac0 tx=0x7ff4f4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.917+0000 7ff4fd7fa700 1 -- 172.21.15.72:0/1039421231 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4f4003970 con 0x7ff500109080 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.917+0000 7ff4fd7fa700 1 -- 172.21.15.72:0/1039421231 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff4f4003b10 con 0x7ff500109080 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.917+0000 7ff5064cc700 1 -- 172.21.15.72:0/1039421231 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50019cd00 con 0x7ff500109080 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.917+0000 7ff4fd7fa700 1 -- 172.21.15.72:0/1039421231 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4f4024610 con 0x7ff500109080 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.917+0000 7ff5064cc700 1 -- 172.21.15.72:0/1039421231 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50019f990 con 0x7ff500109080 2024-04-25T13:03:08.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.917+0000 7ff4fd7fa700 1 -- 172.21.15.72:0/1039421231 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff4f4024c20 con 0x7ff500109080 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.917+0000 7ff4fd7fa700 1 -- 172.21.15.72:0/1039421231 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4f4019700 con 0x7ff500109080 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.918+0000 7ff5064cc700 1 -- 172.21.15.72:0/1039421231 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5000640a0 con 0x7ff500109080 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.920+0000 7ff4fd7fa700 1 -- 172.21.15.72:0/1039421231 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff4f4043e50 con 0x7ff500109080 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.990+0000 7ff5064cc700 1 -- 172.21.15.72:0/1039421231 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff50019fb60 con 0x7ff500109080 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.991+0000 7ff4fd7fa700 1 -- 172.21.15.72:0/1039421231 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff4f4032030 con 0x7ff500109080 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.992+0000 7ff5064cc700 1 -- 172.21.15.72:0/1039421231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 msgr2=0x7ff50019c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.992+0000 7ff5064cc700 1 --2- 172.21.15.72:0/1039421231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50019c4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4011ac0 tx=0x7ff4f4011ba0 comp rx=0 tx=0).stop 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.992+0000 7ff5064cc700 1 -- 172.21.15.72:0/1039421231 shutdown_connections 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.992+0000 7ff5064cc700 1 --2- 172.21.15.72:0/1039421231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff500109080 0x7ff50019c4d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.992+0000 7ff5064cc700 1 -- 172.21.15.72:0/1039421231 >> 172.21.15.72:0/1039421231 conn(0x7ff500102af0 msgr2=0x7ff5001036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.992+0000 7ff5064cc700 1 -- 172.21.15.72:0/1039421231 shutdown_connections 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:07.992+0000 7ff5064cc700 1 -- 172.21.15.72:0/1039421231 wait complete. 2024-04-25T13:03:08.595 INFO:teuthology.orchestra.run.smithi072.stdout:mgr not available, waiting (2/15)... 2024-04-25T13:03:09.022 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:08 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:08.597+0000 7f6f06110200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:03:09.433 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:09.109+0000 7f6f06110200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:03:09.434 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:09.199+0000 7f6f06110200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:03:09.770 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:09.436+0000 7f6f06110200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:03:09.770 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:09.566+0000 7f6f06110200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:03:09.770 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:09.659+0000 7f6f06110200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:03:09.770 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:09.772+0000 7f6f06110200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:10.522 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:10.067+0000 7f6f06110200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:03:10.940 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:10.669+0000 7f6f06110200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:03:10.940 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:10.887+0000 7f6f06110200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:03:11.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: Activating manager daemon x 2024-04-25T13:03:11.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: mgrmap e2: x(active, starting, since 0.0029441s) 2024-04-25T13:03:11.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: Manager daemon x is now available 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' 2024-04-25T13:03:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:10 smithi072 ceph-mon[29199]: from='mgr.14100 172.21.15.72:0/3597479628' entity='mgr.x' 2024-04-25T13:03:12.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:11 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/946217749' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:03:12.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:11 smithi072 ceph-mon[29199]: mgrmap e3: x(active, since 1.0073s) 2024-04-25T13:03:14.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:13 smithi072 ceph-mon[29199]: mgrmap e4: x(active, since 2s) 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:03:15.277 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:03:15.278 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:03:15.279 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:03:15.280 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:02:57.590496+0000", 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.008+0000 7f13247d8700 1 Processor -- start 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.008+0000 7f13247d8700 1 -- start start 2024-04-25T13:03:15.281 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.008+0000 7f13247d8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c10b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.008+0000 7f13247d8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f131c063d20 con 0x7f131c1090f0 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.009+0000 7f1322574700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c10b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.009+0000 7f1322574700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c10b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35840/0 (socket says 172.21.15.72:35840) 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.009+0000 7f1322574700 1 -- 172.21.15.72:0/678243219 learned_addr learned my addr 172.21.15.72:0/678243219 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.009+0000 7f1322574700 1 -- 172.21.15.72:0/678243219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c10ba20 con 0x7f131c1090f0 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.009+0000 7f1322574700 1 --2- 172.21.15.72:0/678243219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c10b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f13180098a0 tx=0x7f1318009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c81a22a7376fb74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.009+0000 7f1321572700 1 -- 172.21.15.72:0/678243219 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1318004160 con 0x7f131c1090f0 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.009+0000 7f1321572700 1 -- 172.21.15.72:0/678243219 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1318004300 con 0x7f131c1090f0 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.009+0000 7f1321572700 1 -- 172.21.15.72:0/678243219 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1318011af0 con 0x7f131c1090f0 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.010+0000 7f13247d8700 1 -- 172.21.15.72:0/678243219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 msgr2=0x7f131c10b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.010+0000 7f13247d8700 1 --2- 172.21.15.72:0/678243219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c10b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f13180098a0 tx=0x7f1318009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.010+0000 7f13247d8700 1 -- 172.21.15.72:0/678243219 shutdown_connections 2024-04-25T13:03:15.282 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.010+0000 7f13247d8700 1 --2- 172.21.15.72:0/678243219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c10b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.010+0000 7f13247d8700 1 -- 172.21.15.72:0/678243219 >> 172.21.15.72:0/678243219 conn(0x7f131c102b80 msgr2=0x7f131c104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.010+0000 7f13247d8700 1 -- 172.21.15.72:0/678243219 shutdown_connections 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.010+0000 7f13247d8700 1 -- 172.21.15.72:0/678243219 wait complete. 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.010+0000 7f13247d8700 1 Processor -- start 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.010+0000 7f13247d8700 1 -- start start 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f13247d8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f13247d8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f131c10c970 con 0x7f131c1090f0 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f1322574700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f1322574700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35850/0 (socket says 172.21.15.72:35850) 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f1322574700 1 -- 172.21.15.72:0/946217749 learned_addr learned my addr 172.21.15.72:0/946217749 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f1322574700 1 -- 172.21.15.72:0/946217749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c19ca40 con 0x7f131c1090f0 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f1322574700 1 --2- 172.21.15.72:0/946217749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c19c500 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1318011ac0 tx=0x7f1318011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f13137fe700 1 -- 172.21.15.72:0/946217749 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1318003970 con 0x7f131c1090f0 2024-04-25T13:03:15.283 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f13137fe700 1 -- 172.21.15.72:0/946217749 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1318003b10 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f13247d8700 1 -- 172.21.15.72:0/946217749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131c19ccd0 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f13137fe700 1 -- 172.21.15.72:0/946217749 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1318024610 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.011+0000 7f13247d8700 1 -- 172.21.15.72:0/946217749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131c19f9c0 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.012+0000 7f13137fe700 1 -- 172.21.15.72:0/946217749 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7f1318024840 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.012+0000 7f13247d8700 1 -- 172.21.15.72:0/946217749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131c0640a0 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.012+0000 7f13137fe700 1 -- 172.21.15.72:0/946217749 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f131804d4b0 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.014+0000 7f13137fe700 1 -- 172.21.15.72:0/946217749 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f13180172b0 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.085+0000 7f13247d8700 1 -- 172.21.15.72:0/946217749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f131c19fb90 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.085+0000 7f13137fe700 1 -- 172.21.15.72:0/946217749 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f131c19fb90 con 0x7f131c1090f0 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.086+0000 7f13247d8700 1 -- 172.21.15.72:0/946217749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 msgr2=0x7f131c19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.086+0000 7f13247d8700 1 --2- 172.21.15.72:0/946217749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c19c500 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1318011ac0 tx=0x7f1318011ba0 comp rx=0 tx=0).stop 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.086+0000 7f13247d8700 1 -- 172.21.15.72:0/946217749 shutdown_connections 2024-04-25T13:03:15.284 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.086+0000 7f13247d8700 1 --2- 172.21.15.72:0/946217749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f131c1090f0 0x7f131c19c500 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:15.285 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.086+0000 7f13247d8700 1 -- 172.21.15.72:0/946217749 >> 172.21.15.72:0/946217749 conn(0x7f131c102b80 msgr2=0x7f131c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:15.285 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.086+0000 7f13247d8700 1 -- 172.21.15.72:0/946217749 shutdown_connections 2024-04-25T13:03:15.285 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:11.086+0000 7f13247d8700 1 -- 172.21.15.72:0/946217749 wait complete. 2024-04-25T13:03:15.285 INFO:teuthology.orchestra.run.smithi072.stdout:mgr not available, waiting (3/15)... 2024-04-25T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:22 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1475271439' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:03:30.297 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:03:30.297 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:03:30.297 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:03:30.297 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:03:30.297 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 20, 2024-04-25T13:03:30.298 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:03:30.299 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:30.299 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:03:30.299 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:03:30.299 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:30.299 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:03:30.299 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:30.299 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:03:30.299 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:03:30.299 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:03:30.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:03:30.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:03:30.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:03:30.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:30.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:03:30.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:03:30.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:03:30.308 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:02:57.590496+0000", 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:03:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.142+0000 7f6302a4b700 1 Processor -- start 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.142+0000 7f6302a4b700 1 -- start start 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f6302a4b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f6302a4b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc109060 con 0x7f62fc108670 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f62fbfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f62fbfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37740/0 (socket says 172.21.15.72:37740) 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f62fbfff700 1 -- 172.21.15.72:0/1374114584 learned_addr learned my addr 172.21.15.72:0/1374114584 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f62fbfff700 1 -- 172.21.15.72:0/1374114584 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc109870 con 0x7f62fc108670 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f62fbfff700 1 --2- 172.21.15.72:0/1374114584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc108a90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0098a0 tx=0x7f62ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c0d2367d4ff981 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f62faffd700 1 -- 172.21.15.72:0/1374114584 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec004160 con 0x7f62fc108670 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f62faffd700 1 -- 172.21.15.72:0/1374114584 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f62ec004300 con 0x7f62fc108670 2024-04-25T13:03:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.143+0000 7f62faffd700 1 -- 172.21.15.72:0/1374114584 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec011af0 con 0x7f62fc108670 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.144+0000 7f6302a4b700 1 -- 172.21.15.72:0/1374114584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 msgr2=0x7f62fc108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.144+0000 7f6302a4b700 1 --2- 172.21.15.72:0/1374114584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc108a90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0098a0 tx=0x7f62ec009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.144+0000 7f6302a4b700 1 -- 172.21.15.72:0/1374114584 shutdown_connections 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.144+0000 7f6302a4b700 1 --2- 172.21.15.72:0/1374114584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc108a90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.144+0000 7f6302a4b700 1 -- 172.21.15.72:0/1374114584 >> 172.21.15.72:0/1374114584 conn(0x7f62fc103c30 msgr2=0x7f62fc106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.144+0000 7f6302a4b700 1 -- 172.21.15.72:0/1374114584 shutdown_connections 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.144+0000 7f6302a4b700 1 -- 172.21.15.72:0/1374114584 wait complete. 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f6302a4b700 1 Processor -- start 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f6302a4b700 1 -- start start 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f6302a4b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f6302a4b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc10caa0 con 0x7f62fc108670 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f62fbfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:30.312 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f62fbfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37744/0 (socket says 172.21.15.72:37744) 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f62fbfff700 1 -- 172.21.15.72:0/1475271439 learned_addr learned my addr 172.21.15.72:0/1475271439 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f62fbfff700 1 -- 172.21.15.72:0/1475271439 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc19ec80 con 0x7f62fc108670 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f62fbfff700 1 --2- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc19e740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009fb0 tx=0x7f62ec011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.145+0000 7f62f97fa700 1 -- 172.21.15.72:0/1475271439 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec0036a0 con 0x7f62fc108670 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.146+0000 7f62f97fa700 1 -- 172.21.15.72:0/1475271439 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f62ec01a430 con 0x7f62fc108670 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.146+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62fc19ef10 con 0x7f62fc108670 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.146+0000 7f62f97fa700 1 -- 172.21.15.72:0/1475271439 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec019550 con 0x7f62fc108670 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.146+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62fc1a1c00 con 0x7f62fc108670 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.146+0000 7f62f97fa700 1 -- 172.21.15.72:0/1475271439 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f62ec019850 con 0x7f62fc108670 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.146+0000 7f62f97fa700 1 --2- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f62e4039510 0x7f62e403b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.147+0000 7f62f97fa700 1 -- 172.21.15.72:0/1475271439 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f62ec04d8b0 con 0x7f62fc108670 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.147+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62fc0640a0 con 0x7f62fc108670 2024-04-25T13:03:30.313 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.147+0000 7f62fb7fe700 1 --2- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f62e4039510 0x7f62e403b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.147+0000 7f62fb7fe700 1 --2- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f62e4039510 0x7f62e403b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62f0006fd0 tx=0x7f62f0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.152+0000 7f62f97fa700 1 -- 172.21.15.72:0/1475271439 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62ec023c90 con 0x7f62fc108670 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.429+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f62fc1a21d0 con 0x7f62fc108670 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.430+0000 7f62f97fa700 1 -- 172.21.15.72:0/1475271439 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f62ec04d2d0 con 0x7f62fc108670 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f62e4039510 msgr2=0x7f62e403b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 --2- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f62e4039510 0x7f62e403b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62f0006fd0 tx=0x7f62f0006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 msgr2=0x7f62fc19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 --2- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc19e740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009fb0 tx=0x7f62ec011940 comp rx=0 tx=0).stop 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 shutdown_connections 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 --2- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f62e4039510 0x7f62e403b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 --2- 172.21.15.72:0/1475271439 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62fc108670 0x7f62fc19e740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 >> 172.21.15.72:0/1475271439 conn(0x7f62fc103c30 msgr2=0x7f62fc104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:30.314 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 shutdown_connections 2024-04-25T13:03:30.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:22.432+0000 7f6302a4b700 1 -- 172.21.15.72:0/1475271439 wait complete. 2024-04-25T13:03:30.315 INFO:teuthology.orchestra.run.smithi072.stdout:mgr is available 2024-04-25T13:03:31.497 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout fsid = d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:03:31.498 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.736+0000 7efefcb6a700 1 Processor -- start 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.736+0000 7efefcb6a700 1 -- start start 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.736+0000 7efefcb6a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef8107e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.736+0000 7efefcb6a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efef8108420 con 0x7efef8107a30 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.736+0000 7efef659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef8107e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:31.499 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.737+0000 7efef659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef8107e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43348/0 (socket says 172.21.15.72:43348) 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.737+0000 7efef659c700 1 -- 172.21.15.72:0/1001393834 learned_addr learned my addr 172.21.15.72:0/1001393834 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.737+0000 7efef659c700 1 -- 172.21.15.72:0/1001393834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef8108750 con 0x7efef8107a30 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.737+0000 7efef659c700 1 --2- 172.21.15.72:0/1001393834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef8107e50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efee80098a0 tx=0x7efee8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1017181af8031af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.737+0000 7efef559a700 1 -- 172.21.15.72:0/1001393834 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efee8004160 con 0x7efef8107a30 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.737+0000 7efef559a700 1 -- 172.21.15.72:0/1001393834 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efee8004300 con 0x7efef8107a30 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.737+0000 7efef559a700 1 -- 172.21.15.72:0/1001393834 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efee8011af0 con 0x7efef8107a30 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.738+0000 7efefcb6a700 1 -- 172.21.15.72:0/1001393834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 msgr2=0x7efef8107e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.738+0000 7efefcb6a700 1 --2- 172.21.15.72:0/1001393834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef8107e50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efee80098a0 tx=0x7efee8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.738+0000 7efefcb6a700 1 -- 172.21.15.72:0/1001393834 shutdown_connections 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.738+0000 7efefcb6a700 1 --2- 172.21.15.72:0/1001393834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef8107e50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.738+0000 7efefcb6a700 1 -- 172.21.15.72:0/1001393834 >> 172.21.15.72:0/1001393834 conn(0x7efef8102270 msgr2=0x7efef8104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.738+0000 7efefcb6a700 1 -- 172.21.15.72:0/1001393834 shutdown_connections 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.738+0000 7efefcb6a700 1 -- 172.21.15.72:0/1001393834 wait complete. 2024-04-25T13:03:31.500 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.738+0000 7efefcb6a700 1 Processor -- start 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efefcb6a700 1 -- start start 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efefcb6a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efefcb6a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efef810b980 con 0x7efef8107a30 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efef659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efef659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43356/0 (socket says 172.21.15.72:43356) 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efef659c700 1 -- 172.21.15.72:0/155647426 learned_addr learned my addr 172.21.15.72:0/155647426 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efef659c700 1 -- 172.21.15.72:0/155647426 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef819cae0 con 0x7efef8107a30 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efef659c700 1 --2- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef819c5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efee8011ac0 tx=0x7efee8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efee77fe700 1 -- 172.21.15.72:0/155647426 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efee8003970 con 0x7efef8107a30 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efee77fe700 1 -- 172.21.15.72:0/155647426 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efee8003b10 con 0x7efef8107a30 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef819cd70 con 0x7efef8107a30 2024-04-25T13:03:31.501 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efee77fe700 1 -- 172.21.15.72:0/155647426 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efee8024610 con 0x7efef8107a30 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.739+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef819fa60 con 0x7efef8107a30 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.740+0000 7efee77fe700 1 -- 172.21.15.72:0/155647426 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7efee8024840 con 0x7efef8107a30 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.740+0000 7efee77fe700 1 --2- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7efee00395b0 0x7efee003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.740+0000 7efee77fe700 1 -- 172.21.15.72:0/155647426 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efee804e830 con 0x7efef8107a30 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.740+0000 7efef5d9b700 1 --2- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7efee00395b0 0x7efee003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.741+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef810bb00 con 0x7efef8107a30 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.741+0000 7efef5d9b700 1 --2- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7efee00395b0 0x7efee003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efeec006fd0 tx=0x7efeec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.746+0000 7efee77fe700 1 -- 172.21.15.72:0/155647426 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efee80172b0 con 0x7efef8107a30 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.943+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7efef802e140 con 0x7efef8107a30 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.944+0000 7efee77fe700 1 -- 172.21.15.72:0/155647426 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7efee800ba20 con 0x7efef8107a30 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7efee00395b0 msgr2=0x7efee003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 --2- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7efee00395b0 0x7efee003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efeec006fd0 tx=0x7efeec006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:31.502 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 msgr2=0x7efef819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:31.503 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 --2- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef819c5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efee8011ac0 tx=0x7efee8011ba0 comp rx=0 tx=0).stop 2024-04-25T13:03:31.503 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 shutdown_connections 2024-04-25T13:03:31.503 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 --2- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7efee00395b0 0x7efee003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:31.503 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 --2- 172.21.15.72:0/155647426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efef8107a30 0x7efef819c5a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:31.503 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 >> 172.21.15.72:0/155647426 conn(0x7efef8102270 msgr2=0x7efef8103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:31.503 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 shutdown_connections 2024-04-25T13:03:31.503 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:30.947+0000 7efefcb6a700 1 -- 172.21.15.72:0/155647426 wait complete. 2024-04-25T13:03:31.503 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling cephadm module... 2024-04-25T13:03:32.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:31 smithi072 ceph-mon[29199]: pgmap v2: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:03:32.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:31 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/155647426' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:03:33.208 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:32 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2848269059' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T13:03:33.208 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:32 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: ignoring --setuser ceph since I am not root 2024-04-25T13:03:33.208 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:32 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: ignoring --setgroup ceph since I am not root 2024-04-25T13:03:33.208 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:33 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:33.077+0000 7f9c2149f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:03:33.478 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:33 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:33.260+0000 7f9c2149f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:33.478 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:33 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:33.362+0000 7f9c2149f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:03:33.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.907+0000 7f8eb1514700 1 Processor -- start 2024-04-25T13:03:33.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.907+0000 7f8eb1514700 1 -- start start 2024-04-25T13:03:33.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.907+0000 7f8eb1514700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac1072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:33.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.907+0000 7f8eb1514700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac07ce70 con 0x7f8eac106eb0 2024-04-25T13:03:33.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.908+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac1072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:33.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.908+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac1072d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43366/0 (socket says 172.21.15.72:43366) 2024-04-25T13:03:33.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.908+0000 7f8eaaffd700 1 -- 172.21.15.72:0/2553569030 learned_addr learned my addr 172.21.15.72:0/2553569030 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:33.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.908+0000 7f8eaaffd700 1 -- 172.21.15.72:0/2553569030 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eac107810 con 0x7f8eac106eb0 2024-04-25T13:03:33.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.908+0000 7f8eaaffd700 1 --2- 172.21.15.72:0/2553569030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac1072d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0098a0 tx=0x7f8e9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eee35a8f652b7e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.908+0000 7f8eaa7fc700 1 -- 172.21.15.72:0/2553569030 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c004160 con 0x7f8eac106eb0 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.908+0000 7f8eaa7fc700 1 -- 172.21.15.72:0/2553569030 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e9c004300 con 0x7f8eac106eb0 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.908+0000 7f8eaa7fc700 1 -- 172.21.15.72:0/2553569030 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c011af0 con 0x7f8eac106eb0 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.909+0000 7f8eb1514700 1 -- 172.21.15.72:0/2553569030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 msgr2=0x7f8eac1072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.909+0000 7f8eb1514700 1 --2- 172.21.15.72:0/2553569030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac1072d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0098a0 tx=0x7f8e9c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.909+0000 7f8eb1514700 1 -- 172.21.15.72:0/2553569030 shutdown_connections 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.909+0000 7f8eb1514700 1 --2- 172.21.15.72:0/2553569030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac1072d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.909+0000 7f8eb1514700 1 -- 172.21.15.72:0/2553569030 >> 172.21.15.72:0/2553569030 conn(0x7f8eac102b10 msgr2=0x7f8eac104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.909+0000 7f8eb1514700 1 -- 172.21.15.72:0/2553569030 shutdown_connections 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.909+0000 7f8eb1514700 1 -- 172.21.15.72:0/2553569030 wait complete. 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.909+0000 7f8eb1514700 1 Processor -- start 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.909+0000 7f8eb1514700 1 -- start start 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8eb1514700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:33.487 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8eb1514700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1087d0 con 0x7f8eac106eb0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43380/0 (socket says 172.21.15.72:43380) 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8eaaffd700 1 -- 172.21.15.72:0/2848269059 learned_addr learned my addr 172.21.15.72:0/2848269059 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8eaaffd700 1 -- 172.21.15.72:0/2848269059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eac19ec40 con 0x7f8eac106eb0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8eaaffd700 1 --2- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac19e700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009fb0 tx=0x7f8e9c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8ea8ff9700 1 -- 172.21.15.72:0/2848269059 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c0036a0 con 0x7f8eac106eb0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8ea8ff9700 1 -- 172.21.15.72:0/2848269059 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e9c01a430 con 0x7f8eac106eb0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eac19eed0 con 0x7f8eac106eb0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8ea8ff9700 1 -- 172.21.15.72:0/2848269059 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c019550 con 0x7f8eac106eb0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.910+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eac1a1bc0 con 0x7f8eac106eb0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.911+0000 7f8ea8ff9700 1 -- 172.21.15.72:0/2848269059 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f8e9c003840 con 0x7f8eac106eb0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.911+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eac0640a0 con 0x7f8eac106eb0 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.911+0000 7f8ea8ff9700 1 --2- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f8e98039510 0x7f8e9803b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:33.488 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.911+0000 7f8ea8ff9700 1 -- 172.21.15.72:0/2848269059 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8e9c04d8a0 con 0x7f8eac106eb0 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.912+0000 7f8ea3fff700 1 --2- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f8e98039510 0x7f8e9803b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.912+0000 7f8ea3fff700 1 --2- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f8e98039510 0x7f8e9803b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:31.917+0000 7f8ea8ff9700 1 -- 172.21.15.72:0/2848269059 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e9c023e00 con 0x7f8eac106eb0 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.168+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f8eac0008d0 con 0x7f8eac106eb0 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.898+0000 7f8ea8ff9700 1 -- 172.21.15.72:0/2848269059 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f8e9c04d2c0 con 0x7f8eac106eb0 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.900+0000 7f8ea8ff9700 1 -- 172.21.15.72:0/2848269059 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f8e9c019850 con 0x7f8eac106eb0 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f8e98039510 msgr2=0x7f8e9803b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 --2- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f8e98039510 0x7f8e9803b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 msgr2=0x7f8eac19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 --2- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac19e700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009fb0 tx=0x7f8e9c011940 comp rx=0 tx=0).stop 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 shutdown_connections 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 --2- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f8e98039510 0x7f8e9803b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:33.489 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 --2- 172.21.15.72:0/2848269059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8eac106eb0 0x7f8eac19e700 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:33.490 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 >> 172.21.15.72:0/2848269059 conn(0x7f8eac102b10 msgr2=0x7f8eac1953c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:33.490 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 shutdown_connections 2024-04-25T13:03:33.490 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:32.902+0000 7f8eb1514700 1 -- 172.21.15.72:0/2848269059 wait complete. 2024-04-25T13:03:33.765 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:33 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:33.678+0000 7f9c2149f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:03:33.765 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:33 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:33.765+0000 7f9c2149f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:03:34.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:33 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2848269059' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T13:03:34.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:33 smithi072 ceph-mon[29199]: mgrmap e5: x(active, since 22s) 2024-04-25T13:03:34.775 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:03:34.775 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T13:03:34.775 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.925+0000 7f224ff32700 1 Processor -- start 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.925+0000 7f224ff32700 1 -- start start 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.925+0000 7f224ff32700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f2248108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.925+0000 7f224ff32700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2248109050 con 0x7f2248108660 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.926+0000 7f224dcce700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f2248108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.926+0000 7f224dcce700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f2248108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43404/0 (socket says 172.21.15.72:43404) 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.926+0000 7f224dcce700 1 -- 172.21.15.72:0/374062679 learned_addr learned my addr 172.21.15.72:0/374062679 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.926+0000 7f224dcce700 1 -- 172.21.15.72:0/374062679 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2248109810 con 0x7f2248108660 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.926+0000 7f224dcce700 1 --2- 172.21.15.72:0/374062679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f2248108a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22380098a0 tx=0x7f2238009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a960345eb09e39be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:34.776 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.926+0000 7f224cccc700 1 -- 172.21.15.72:0/374062679 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238004160 con 0x7f2248108660 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.926+0000 7f224cccc700 1 -- 172.21.15.72:0/374062679 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2238004300 con 0x7f2248108660 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.926+0000 7f224cccc700 1 -- 172.21.15.72:0/374062679 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238011af0 con 0x7f2248108660 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.927+0000 7f224ff32700 1 -- 172.21.15.72:0/374062679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 msgr2=0x7f2248108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.927+0000 7f224ff32700 1 --2- 172.21.15.72:0/374062679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f2248108a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22380098a0 tx=0x7f2238009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.927+0000 7f224ff32700 1 -- 172.21.15.72:0/374062679 shutdown_connections 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.927+0000 7f224ff32700 1 --2- 172.21.15.72:0/374062679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f2248108a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.927+0000 7f224ff32700 1 -- 172.21.15.72:0/374062679 >> 172.21.15.72:0/374062679 conn(0x7f2248103c40 msgr2=0x7f2248106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.927+0000 7f224ff32700 1 -- 172.21.15.72:0/374062679 shutdown_connections 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.927+0000 7f224ff32700 1 -- 172.21.15.72:0/374062679 wait complete. 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.928+0000 7f224ff32700 1 Processor -- start 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.928+0000 7f224ff32700 1 -- start start 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.928+0000 7f224ff32700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f224807eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:34.777 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.928+0000 7f224ff32700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f224810ca40 con 0x7f2248108660 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.928+0000 7f224dcce700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f224807eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.928+0000 7f224dcce700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f224807eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43416/0 (socket says 172.21.15.72:43416) 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.928+0000 7f224dcce700 1 -- 172.21.15.72:0/2762704200 learned_addr learned my addr 172.21.15.72:0/2762704200 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.928+0000 7f224dcce700 1 -- 172.21.15.72:0/2762704200 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224807f0b0 con 0x7f2248108660 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.929+0000 7f224dcce700 1 --2- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f224807eb70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2238009fb0 tx=0x7f2238011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.929+0000 7f223effd700 1 -- 172.21.15.72:0/2762704200 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22380036a0 con 0x7f2248108660 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.929+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224807d2a0 con 0x7f2248108660 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.929+0000 7f223effd700 1 -- 172.21.15.72:0/2762704200 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f223801a430 con 0x7f2248108660 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.929+0000 7f223effd700 1 -- 172.21.15.72:0/2762704200 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238019550 con 0x7f2248108660 2024-04-25T13:03:34.778 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.929+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224807d7e0 con 0x7f2248108660 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.930+0000 7f223effd700 1 -- 172.21.15.72:0/2762704200 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f2238019850 con 0x7f2248108660 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.930+0000 7f223effd700 1 --2- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f2234039560 0x7f223403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.930+0000 7f223effd700 1 -- 172.21.15.72:0/2762704200 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f223804d8f0 con 0x7f2248108660 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.930+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22480640a0 con 0x7f2248108660 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.930+0000 7f224d4cd700 1 -- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f2234039560 msgr2=0x7f223403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2706752188 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.930+0000 7f224d4cd700 1 --2- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f2234039560 0x7f223403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:33.935+0000 7f223effd700 1 -- 172.21.15.72:0/2762704200 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2238016dd0 con 0x7f2248108660 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.131+0000 7f224d4cd700 1 -- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f2234039560 msgr2=0x7f223403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2706752188 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.131+0000 7f224d4cd700 1 --2- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f2234039560 0x7f223403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.231+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f22481a2570 con 0x7f2248108660 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.232+0000 7f223effd700 1 -- 172.21.15.72:0/2762704200 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f223804d310 con 0x7f2248108660 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.235+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f2234039560 msgr2=0x7f223403ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:03:34.779 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.235+0000 7f224ff32700 1 --2- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f2234039560 0x7f223403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.235+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 msgr2=0x7f224807eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.235+0000 7f224ff32700 1 --2- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f224807eb70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2238009fb0 tx=0x7f2238011940 comp rx=0 tx=0).stop 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.235+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 shutdown_connections 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.235+0000 7f224ff32700 1 --2- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f2234039560 0x7f223403ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.235+0000 7f224ff32700 1 --2- 172.21.15.72:0/2762704200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2248108660 0x7f224807eb70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.235+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 >> 172.21.15.72:0/2762704200 conn(0x7f2248103c40 msgr2=0x7f2248104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.236+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 shutdown_connections 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:34.236+0000 7f224ff32700 1 -- 172.21.15.72:0/2762704200 wait complete. 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for the mgr to restart... 2024-04-25T13:03:34.780 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr epoch 5... 2024-04-25T13:03:35.146 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:34 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2762704200' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:03:35.146 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:34 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:34.975+0000 7f9c2149f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:03:35.146 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:35 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:35.147+0000 7f9c2149f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:35.522 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:35 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:35.314+0000 7f9c2149f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:03:35.896 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:35 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:35.603+0000 7f9c2149f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:03:35.896 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:35 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:35.696+0000 7f9c2149f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:03:35.896 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:35 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:35.897+0000 7f9c2149f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:03:36.247 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:36 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:36.096+0000 7f9c2149f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:37.022 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:36 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:36.690+0000 7f9c2149f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:03:37.409 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:37 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:37.111+0000 7f9c2149f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:03:37.409 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:37 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:37.207+0000 7f9c2149f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:03:37.410 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:37 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:37.410+0000 7f9c2149f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:03:38.259 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:37 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:37.928+0000 7f9c2149f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:03:38.259 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:38.020+0000 7f9c2149f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:03:38.522 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:38.260+0000 7f9c2149f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:03:38.522 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:38.391+0000 7f9c2149f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:03:38.522 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:38.487+0000 7f9c2149f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:03:38.925 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:38.603+0000 7f9c2149f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:38.925 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:38.926+0000 7f9c2149f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:03:39.772 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:39.521+0000 7f9c2149f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:03:39.772 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:03:39.741+0000 7f9c2149f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:03:40.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: Active manager daemon x restarted 2024-04-25T13:03:40.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: Activating manager daemon x 2024-04-25T13:03:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T13:03:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: mgrmap e6: x(active, starting, since 0.00334254s) 2024-04-25T13:03:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:03:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:03:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:03:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:03:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:03:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: Manager daemon x is now available 2024-04-25T13:03:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:40.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:40.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:40.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:40.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:39 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:03:41.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:40 smithi072 ceph-mon[29199]: Found migration_current of "None". Setting to last migration. 2024-04-25T13:03:41.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:40 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:03:41.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:40 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:41.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:40 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:41.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:40 smithi072 ceph-mon[29199]: mgrmap e7: x(active, since 1.00518s) 2024-04-25T13:03:41.385 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:03:41.385 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T13:03:41.385 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:03:41.385 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:03:41.386 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.043+0000 7f4152f89700 1 Processor -- start 2024-04-25T13:03:41.386 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.043+0000 7f4152f89700 1 -- start start 2024-04-25T13:03:41.386 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.043+0000 7f4152f89700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:41.386 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.043+0000 7f4152f89700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f414c07d8d0 con 0x7f414c07eea0 2024-04-25T13:03:41.386 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.044+0000 7f4150d25700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:41.386 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.044+0000 7f4150d25700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44424/0 (socket says 172.21.15.72:44424) 2024-04-25T13:03:41.386 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.044+0000 7f4150d25700 1 -- 172.21.15.72:0/2881872586 learned_addr learned my addr 172.21.15.72:0/2881872586 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:41.386 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.044+0000 7f4150d25700 1 -- 172.21.15.72:0/2881872586 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c07da50 con 0x7f414c07eea0 2024-04-25T13:03:41.386 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.044+0000 7f4150d25700 1 --2- 172.21.15.72:0/2881872586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c07d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f41400098a0 tx=0x7f4140009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abcd16859be6561 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.044+0000 7f414b7fe700 1 -- 172.21.15.72:0/2881872586 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4140004160 con 0x7f414c07eea0 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.044+0000 7f414b7fe700 1 -- 172.21.15.72:0/2881872586 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4140004300 con 0x7f414c07eea0 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.044+0000 7f414b7fe700 1 -- 172.21.15.72:0/2881872586 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4140011af0 con 0x7f414c07eea0 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.045+0000 7f4152f89700 1 -- 172.21.15.72:0/2881872586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 msgr2=0x7f414c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.045+0000 7f4152f89700 1 --2- 172.21.15.72:0/2881872586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c07d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f41400098a0 tx=0x7f4140009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.045+0000 7f4152f89700 1 -- 172.21.15.72:0/2881872586 shutdown_connections 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.045+0000 7f4152f89700 1 --2- 172.21.15.72:0/2881872586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c07d300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.045+0000 7f4152f89700 1 -- 172.21.15.72:0/2881872586 >> 172.21.15.72:0/2881872586 conn(0x7f414c103cc0 msgr2=0x7f414c1060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.045+0000 7f4152f89700 1 -- 172.21.15.72:0/2881872586 shutdown_connections 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.045+0000 7f4152f89700 1 -- 172.21.15.72:0/2881872586 wait complete. 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4152f89700 1 Processor -- start 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4152f89700 1 -- start start 2024-04-25T13:03:41.387 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4152f89700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4152f89700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f414c10eb80 con 0x7f414c07eea0 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4150d25700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4150d25700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44428/0 (socket says 172.21.15.72:44428) 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4150d25700 1 -- 172.21.15.72:0/74527998 learned_addr learned my addr 172.21.15.72:0/74527998 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4150d25700 1 -- 172.21.15.72:0/74527998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c19ecd0 con 0x7f414c07eea0 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4150d25700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c19e790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4140009fb0 tx=0x7f4140011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41400036a0 con 0x7f414c07eea0 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f414001a430 con 0x7f414c07eea0 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4152f89700 1 -- 172.21.15.72:0/74527998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c19ef60 con 0x7f414c07eea0 2024-04-25T13:03:41.388 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.046+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4140019550 con 0x7f414c07eea0 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.047+0000 7f4152f89700 1 -- 172.21.15.72:0/74527998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c1a1c50 con 0x7f414c07eea0 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.047+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f4140019850 con 0x7f414c07eea0 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.048+0000 7f4149ffb700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 0x7f413c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.048+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 --> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f413c03c130 con 0x7f413c039560 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.048+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f414004d930 con 0x7f414c07eea0 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.048+0000 7f414bfff700 1 -- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 msgr2=0x7f413c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2706752188 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.048+0000 7f414bfff700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 0x7f413c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.249+0000 7f414bfff700 1 -- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 msgr2=0x7f413c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2706752188 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.249+0000 7f414bfff700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 0x7f413c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.650+0000 7f414bfff700 1 -- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 msgr2=0x7f413c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2706752188 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:36.650+0000 7f414bfff700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 0x7f413c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:37.451+0000 7f414bfff700 1 -- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 msgr2=0x7f413c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2706752188 2024-04-25T13:03:41.389 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:37.451+0000 7f414bfff700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 0x7f413c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:39.052+0000 7f414bfff700 1 -- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 msgr2=0x7f413c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2706752188 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:39.052+0000 7f414bfff700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 0x7f413c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:39.744+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f4140019b90 con 0x7f414c07eea0 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:39.744+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 msgr2=0x7f413c03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:39.744+0000 7f4149ffb700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 0x7f413c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.745+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f414004f9a0 con 0x7f414c07eea0 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.745+0000 7f4149ffb700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f413c03ce50 0x7f413c03f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.745+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4140022e70 con 0x7f413c03ce50 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.745+0000 7f414bfff700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f413c03ce50 0x7f413c03f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.746+0000 7f414bfff700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f413c03ce50 0x7f413c03f240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4138003a10 tx=0x7f4138009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.747+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 <== mgr.14120 v2:172.21.15.72:6800/2774697366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f4140022e70 con 0x7f413c03ce50 2024-04-25T13:03:41.390 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.757+0000 7f4152f89700 1 -- 172.21.15.72:0/74527998 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f414c0640a0 con 0x7f413c03ce50 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4149ffb700 1 -- 172.21.15.72:0/74527998 <== mgr.14120 v2:172.21.15.72:6800/2774697366 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f414c0640a0 con 0x7f413c03ce50 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 -- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f413c03ce50 msgr2=0x7f413c03f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f413c03ce50 0x7f413c03f240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4138003a10 tx=0x7f4138009290 comp rx=0 tx=0).stop 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 -- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 msgr2=0x7f414c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c19e790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4140009fb0 tx=0x7f4140011940 comp rx=0 tx=0).stop 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 -- 172.21.15.72:0/74527998 shutdown_connections 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f413c03ce50 0x7f413c03f240 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f414c07eea0 0x7f414c19e790 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 --2- 172.21.15.72:0/74527998 >> [v2:172.21.15.72:6800/2706752188,v1:172.21.15.72:6801/2706752188] conn(0x7f413c039560 0x7f413c03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 -- 172.21.15.72:0/74527998 >> 172.21.15.72:0/74527998 conn(0x7f414c103cc0 msgr2=0x7f414c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 -- 172.21.15.72:0/74527998 shutdown_connections 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:40.758+0000 7f4152f89700 1 -- 172.21.15.72:0/74527998 wait complete. 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:mgr epoch 5 is available 2024-04-25T13:03:41.391 INFO:teuthology.orchestra.run.smithi072.stdout:Setting orchestrator backend to cephadm... 2024-04-25T13:03:42.019 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:41 smithi072 ceph-mon[29199]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T13:03:42.019 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:41 smithi072 ceph-mon[29199]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T13:03:42.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.790+0000 7f40ed103700 1 Processor -- start 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.790+0000 7f40ed103700 1 -- start start 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.790+0000 7f40ed103700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e810b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.790+0000 7f40ed103700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f40e807ce70 con 0x7f40e81090c0 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.790+0000 7f40e6d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e810b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.790+0000 7f40e6d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e810b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44486/0 (socket says 172.21.15.72:44486) 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.790+0000 7f40e6d9d700 1 -- 172.21.15.72:0/1508446178 learned_addr learned my addr 172.21.15.72:0/1508446178 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40e6d9d700 1 -- 172.21.15.72:0/1508446178 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e810b9f0 con 0x7f40e81090c0 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40e6d9d700 1 --2- 172.21.15.72:0/1508446178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e810b4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40d80098a0 tx=0x7f40d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f2d6377bd4f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40e5d9b700 1 -- 172.21.15.72:0/1508446178 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40d8004160 con 0x7f40e81090c0 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40e5d9b700 1 -- 172.21.15.72:0/1508446178 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f40d8004300 con 0x7f40e81090c0 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40e5d9b700 1 -- 172.21.15.72:0/1508446178 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40d8011af0 con 0x7f40e81090c0 2024-04-25T13:03:42.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40ed103700 1 -- 172.21.15.72:0/1508446178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 msgr2=0x7f40e810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40ed103700 1 --2- 172.21.15.72:0/1508446178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e810b4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40d80098a0 tx=0x7f40d8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40ed103700 1 -- 172.21.15.72:0/1508446178 shutdown_connections 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40ed103700 1 --2- 172.21.15.72:0/1508446178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e810b4b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.791+0000 7f40ed103700 1 -- 172.21.15.72:0/1508446178 >> 172.21.15.72:0/1508446178 conn(0x7f40e8102b10 msgr2=0x7f40e8104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.792+0000 7f40ed103700 1 -- 172.21.15.72:0/1508446178 shutdown_connections 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.792+0000 7f40ed103700 1 -- 172.21.15.72:0/1508446178 wait complete. 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.792+0000 7f40ed103700 1 Processor -- start 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.792+0000 7f40ed103700 1 -- start start 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40ed103700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e8108940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40ed103700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f40e810c940 con 0x7f40e81090c0 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40e6d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e8108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40e6d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e8108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44488/0 (socket says 172.21.15.72:44488) 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40e6d9d700 1 -- 172.21.15.72:0/1686855335 learned_addr learned my addr 172.21.15.72:0/1686855335 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40e6d9d700 1 -- 172.21.15.72:0/1686855335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e8108e80 con 0x7f40e81090c0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40e6d9d700 1 --2- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e8108940 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40d8011ac0 tx=0x7f40d8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40cffff700 1 -- 172.21.15.72:0/1686855335 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40d8003970 con 0x7f40e81090c0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e8107070 con 0x7f40e81090c0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40cffff700 1 -- 172.21.15.72:0/1686855335 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f40d8003b10 con 0x7f40e81090c0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40cffff700 1 -- 172.21.15.72:0/1686855335 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40d8024610 con 0x7f40e81090c0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.793+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e81075b0 con 0x7f40e81090c0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.794+0000 7f40cffff700 1 -- 172.21.15.72:0/1686855335 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f40d8024890 con 0x7f40e81090c0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.794+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e80640a0 con 0x7f40e81090c0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.795+0000 7f40cffff700 1 --2- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f40d0039490 0x7f40d003b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.795+0000 7f40cffff700 1 -- 172.21.15.72:0/1686855335 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f40d804d790 con 0x7f40e81090c0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.795+0000 7f40e659c700 1 --2- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f40d0039490 0x7f40d003b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.795+0000 7f40e659c700 1 --2- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f40d0039490 0x7f40d003b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40dc006fd0 tx=0x7f40dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:42.549 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.800+0000 7f40cffff700 1 -- 172.21.15.72:0/1686855335 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40d80172b0 con 0x7f40e81090c0 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:41.805+0000 7f40cffff700 1 -- 172.21.15.72:0/1686855335 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f40d802d910 con 0x7f40e81090c0 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.020+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f40e8103de0 con 0x7f40d0039490 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.025+0000 7f40cffff700 1 -- 172.21.15.72:0/1686855335 <== mgr.14120 v2:172.21.15.72:6800/2774697366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f40e8103de0 con 0x7f40d0039490 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f40d0039490 msgr2=0x7f40d003b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 --2- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f40d0039490 0x7f40d003b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40dc006fd0 tx=0x7f40dc006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 msgr2=0x7f40e8108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 --2- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e8108940 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40d8011ac0 tx=0x7f40d8011ba0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 shutdown_connections 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 --2- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f40d0039490 0x7f40d003b950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 --2- 172.21.15.72:0/1686855335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e81090c0 0x7f40e8108940 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 >> 172.21.15.72:0/1686855335 conn(0x7f40e8102b10 msgr2=0x7f40e81036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:42.550 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 shutdown_connections 2024-04-25T13:03:42.551 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.027+0000 7f40ed103700 1 -- 172.21.15.72:0/1686855335 wait complete. 2024-04-25T13:03:42.919 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:42 smithi072 ceph-mon[29199]: mgrmap e8: x(active, since 2s) 2024-04-25T13:03:42.919 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:42 smithi072 ceph-mon[29199]: [25/Apr/2024:13:03:41] ENGINE Bus STARTING 2024-04-25T13:03:42.919 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:42 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:42.919 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:42 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:42.919 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:42 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:43.679 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T13:03:43.679 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.920+0000 7f60b0dad700 1 Processor -- start 2024-04-25T13:03:43.679 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.920+0000 7f60b0dad700 1 -- start start 2024-04-25T13:03:43.679 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.920+0000 7f60b0dad700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:43.679 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.920+0000 7f60b0dad700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac109000 con 0x7f60ac108610 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.920+0000 7f60aa59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.920+0000 7f60aa59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44498/0 (socket says 172.21.15.72:44498) 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.920+0000 7f60aa59c700 1 -- 172.21.15.72:0/3296708857 learned_addr learned my addr 172.21.15.72:0/3296708857 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.920+0000 7f60aa59c700 1 -- 172.21.15.72:0/3296708857 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac109810 con 0x7f60ac108610 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.921+0000 7f60aa59c700 1 --2- 172.21.15.72:0/3296708857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac108a30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098a0 tx=0x7f609c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4ea1e077946edf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.921+0000 7f60a959a700 1 -- 172.21.15.72:0/3296708857 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f609c004160 con 0x7f60ac108610 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.921+0000 7f60a959a700 1 -- 172.21.15.72:0/3296708857 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f609c004300 con 0x7f60ac108610 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.921+0000 7f60a959a700 1 -- 172.21.15.72:0/3296708857 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f609c011af0 con 0x7f60ac108610 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.921+0000 7f60b0dad700 1 -- 172.21.15.72:0/3296708857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 msgr2=0x7f60ac108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.921+0000 7f60b0dad700 1 --2- 172.21.15.72:0/3296708857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac108a30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098a0 tx=0x7f609c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.922+0000 7f60b0dad700 1 -- 172.21.15.72:0/3296708857 shutdown_connections 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.922+0000 7f60b0dad700 1 --2- 172.21.15.72:0/3296708857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac108a30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.680 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.922+0000 7f60b0dad700 1 -- 172.21.15.72:0/3296708857 >> 172.21.15.72:0/3296708857 conn(0x7f60ac103bf0 msgr2=0x7f60ac106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.922+0000 7f60b0dad700 1 -- 172.21.15.72:0/3296708857 shutdown_connections 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.922+0000 7f60b0dad700 1 -- 172.21.15.72:0/3296708857 wait complete. 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.922+0000 7f60b0dad700 1 Processor -- start 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.922+0000 7f60b0dad700 1 -- start start 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.922+0000 7f60b0dad700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f60b0dad700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10ca40 con 0x7f60ac108610 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f60aa59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f60aa59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44514/0 (socket says 172.21.15.72:44514) 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f60aa59c700 1 -- 172.21.15.72:0/3364568652 learned_addr learned my addr 172.21.15.72:0/3364568652 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f60aa59c700 1 -- 172.21.15.72:0/3364568652 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac19ca60 con 0x7f60ac108610 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f60aa59c700 1 --2- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac19c520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f609c009fb0 tx=0x7f609c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f609b7fe700 1 -- 172.21.15.72:0/3364568652 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f609c0036a0 con 0x7f60ac108610 2024-04-25T13:03:43.681 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f609b7fe700 1 -- 172.21.15.72:0/3364568652 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f609c01a430 con 0x7f60ac108610 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac19ccf0 con 0x7f60ac108610 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f609b7fe700 1 -- 172.21.15.72:0/3364568652 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f609c019550 con 0x7f60ac108610 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.923+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac19f9e0 con 0x7f60ac108610 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.924+0000 7f609b7fe700 1 -- 172.21.15.72:0/3364568652 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f609c019800 con 0x7f60ac108610 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.924+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac10cbc0 con 0x7f60ac108610 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.924+0000 7f609b7fe700 1 --2- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f60940395b0 0x7f609403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.924+0000 7f609b7fe700 1 -- 172.21.15.72:0/3364568652 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f609c04d890 con 0x7f60ac108610 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.924+0000 7f60a9d9b700 1 --2- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f60940395b0 0x7f609403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.925+0000 7f60a9d9b700 1 --2- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f60940395b0 0x7f609403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006fd0 tx=0x7f60a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:42.930+0000 7f609b7fe700 1 -- 172.21.15.72:0/3364568652 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f609c023c90 con 0x7f60ac108610 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.130+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f60ac104e60 con 0x7f60940395b0 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.130+0000 7f609b7fe700 1 -- 172.21.15.72:0/3364568652 <== mgr.14120 v2:172.21.15.72:6800/2774697366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f60ac104e60 con 0x7f60940395b0 2024-04-25T13:03:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.132+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f60940395b0 msgr2=0x7f609403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.132+0000 7f60b0dad700 1 --2- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f60940395b0 0x7f609403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006fd0 tx=0x7f60a0006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.132+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 msgr2=0x7f60ac19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.132+0000 7f60b0dad700 1 --2- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac19c520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f609c009fb0 tx=0x7f609c011940 comp rx=0 tx=0).stop 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.133+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 shutdown_connections 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.133+0000 7f60b0dad700 1 --2- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f60940395b0 0x7f609403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.133+0000 7f60b0dad700 1 --2- 172.21.15.72:0/3364568652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ac108610 0x7f60ac19c520 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.133+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 >> 172.21.15.72:0/3364568652 conn(0x7f60ac103bf0 msgr2=0x7f60ac104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.133+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 shutdown_connections 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:43.133+0000 7f60b0dad700 1 -- 172.21.15.72:0/3364568652 wait complete. 2024-04-25T13:03:43.683 INFO:teuthology.orchestra.run.smithi072.stdout:Generating ssh key... 2024-04-25T13:03:44.093 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:43 smithi072 ceph-mon[29199]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:44.093 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:43 smithi072 ceph-mon[29199]: [25/Apr/2024:13:03:42] ENGINE Serving on https://172.21.15.72:7150 2024-04-25T13:03:44.093 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:43 smithi072 ceph-mon[29199]: [25/Apr/2024:13:03:42] ENGINE Serving on http://172.21.15.72:8765 2024-04-25T13:03:44.093 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:43 smithi072 ceph-mon[29199]: [25/Apr/2024:13:03:42] ENGINE Bus STARTED 2024-04-25T13:03:45.140 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-mon[29199]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: Generating public/private rsa key pair. 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: Your identification has been saved in /tmp/tmpfjd5lw5v/key. 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: Your public key has been saved in /tmp/tmpfjd5lw5v/key.pub. 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: The key fingerprint is: 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: SHA256:fL41fofu27T4GfNusE6azqvyA8+91REjqg7UKoDVHi8 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: The key's randomart image is: 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: +---[RSA 3072]----+ 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: | | 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: | . | 2024-04-25T13:03:45.141 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: | . o . o | 2024-04-25T13:03:45.142 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: | o . o.. . . o| 2024-04-25T13:03:45.142 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: | . . E oS... . | 2024-04-25T13:03:45.142 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: | . o oo. ...| 2024-04-25T13:03:45.142 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: | . o =..o o*o| 2024-04-25T13:03:45.142 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: | . + +=.*=oO| 2024-04-25T13:03:45.142 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: | +++OO*O+| 2024-04-25T13:03:45.142 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:03:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: +----[SHA256]-----+ 2024-04-25T13:03:45.409 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.107+0000 7fb92ee06700 1 Processor -- start 2024-04-25T13:03:45.409 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.107+0000 7fb92ee06700 1 -- start start 2024-04-25T13:03:45.409 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb92ee06700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb928106d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:45.409 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb92ee06700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281072e0 con 0x7fb9281068f0 2024-04-25T13:03:45.409 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb92cba2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb928106d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:45.409 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb92cba2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb928106d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44522/0 (socket says 172.21.15.72:44522) 2024-04-25T13:03:45.409 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb92cba2700 1 -- 172.21.15.72:0/4104679156 learned_addr learned my addr 172.21.15.72:0/4104679156 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:45.409 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb92cba2700 1 -- 172.21.15.72:0/4104679156 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb928109880 con 0x7fb9281068f0 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb92cba2700 1 --2- 172.21.15.72:0/4104679156 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb928106d10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098a0 tx=0x7fb91c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9332f771a195ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb9277fe700 1 -- 172.21.15.72:0/4104679156 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c004160 con 0x7fb9281068f0 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb9277fe700 1 -- 172.21.15.72:0/4104679156 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb91c004300 con 0x7fb9281068f0 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.108+0000 7fb9277fe700 1 -- 172.21.15.72:0/4104679156 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c011af0 con 0x7fb9281068f0 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.109+0000 7fb92ee06700 1 -- 172.21.15.72:0/4104679156 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 msgr2=0x7fb928106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.109+0000 7fb92ee06700 1 --2- 172.21.15.72:0/4104679156 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb928106d10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098a0 tx=0x7fb91c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.109+0000 7fb92ee06700 1 -- 172.21.15.72:0/4104679156 shutdown_connections 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.109+0000 7fb92ee06700 1 --2- 172.21.15.72:0/4104679156 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb928106d10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.109+0000 7fb92ee06700 1 -- 172.21.15.72:0/4104679156 >> 172.21.15.72:0/4104679156 conn(0x7fb928102240 msgr2=0x7fb928104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.109+0000 7fb92ee06700 1 -- 172.21.15.72:0/4104679156 shutdown_connections 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.109+0000 7fb92ee06700 1 -- 172.21.15.72:0/4104679156 wait complete. 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92ee06700 1 Processor -- start 2024-04-25T13:03:45.410 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92ee06700 1 -- start start 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92ee06700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb92819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92ee06700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb92810cab0 con 0x7fb9281068f0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92cba2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb92819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92cba2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb92819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44534/0 (socket says 172.21.15.72:44534) 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92cba2700 1 -- 172.21.15.72:0/331132890 learned_addr learned my addr 172.21.15.72:0/331132890 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92cba2700 1 -- 172.21.15.72:0/331132890 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92819cab0 con 0x7fb9281068f0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92cba2700 1 --2- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb92819c570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009fb0 tx=0x7fb91c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb925ffb700 1 -- 172.21.15.72:0/331132890 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c0036a0 con 0x7fb9281068f0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb925ffb700 1 -- 172.21.15.72:0/331132890 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb91c01a430 con 0x7fb9281068f0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92819cd40 con 0x7fb9281068f0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.110+0000 7fb925ffb700 1 -- 172.21.15.72:0/331132890 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c019550 con 0x7fb9281068f0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.111+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92819fa30 con 0x7fb9281068f0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.111+0000 7fb925ffb700 1 -- 172.21.15.72:0/331132890 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb91c01a8e0 con 0x7fb9281068f0 2024-04-25T13:03:45.411 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.112+0000 7fb925ffb700 1 --2- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fb918039560 0x7fb91803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.112+0000 7fb925ffb700 1 -- 172.21.15.72:0/331132890 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb91c04d920 con 0x7fb9281068f0 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.112+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9280640a0 con 0x7fb9281068f0 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.112+0000 7fb927fff700 1 --2- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fb918039560 0x7fb91803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.112+0000 7fb927fff700 1 --2- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fb918039560 0x7fb91803ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb914006fd0 tx=0x7fb914006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.117+0000 7fb925ffb700 1 -- 172.21.15.72:0/331132890 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb91c00fda0 con 0x7fb9281068f0 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.317+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fb9281034a0 con 0x7fb918039560 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.840+0000 7fb925ffb700 1 -- 172.21.15.72:0/331132890 <== mgr.14120 v2:172.21.15.72:6800/2774697366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb9281034a0 con 0x7fb918039560 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.842+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fb918039560 msgr2=0x7fb91803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.842+0000 7fb92ee06700 1 --2- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fb918039560 0x7fb91803ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb914006fd0 tx=0x7fb914006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.842+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 msgr2=0x7fb92819c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.842+0000 7fb92ee06700 1 --2- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb92819c570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009fb0 tx=0x7fb91c011940 comp rx=0 tx=0).stop 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.842+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 shutdown_connections 2024-04-25T13:03:45.412 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.842+0000 7fb92ee06700 1 --2- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fb918039560 0x7fb91803ba20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:45.413 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.842+0000 7fb92ee06700 1 --2- 172.21.15.72:0/331132890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb9281068f0 0x7fb92819c570 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:45.413 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.842+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 >> 172.21.15.72:0/331132890 conn(0x7fb928102240 msgr2=0x7fb928102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:45.413 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.843+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 shutdown_connections 2024-04-25T13:03:45.413 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:44.843+0000 7fb92ee06700 1 -- 172.21.15.72:0/331132890 wait complete. 2024-04-25T13:03:46.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:45 smithi072 ceph-mon[29199]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:46.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:45 smithi072 ceph-mon[29199]: Generating ssh key... 2024-04-25T13:03:46.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:45 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:46.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:45 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:46.666 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUs9VyOnU4opZcEE4SjeT91WeK+Rv42I8BuXDuM+oRAd/wcIDdvhe5va+qs+XgU9KifRWF+VKRuYD6wKBXl+s/dTFkTY4zpQGr+x/02mHBdGCxcyWv0mryiuneX1G8wB1GruAJGV2waXkFjIWjTFU8fh8yhGzr/+fck6I5hB8mlugLA/0ODuz1nZeO1cOqYtOG+zi892orWwNdTejFoUSvTS/SZctonxdj+YjMwaOnW1AXixGT48u2C5XfFpxZuRVR8DOsGfRYX3Jk2vO7wkjg1BPZ0FXoULFXalEE+RfWrG9IrcMs+nMr+2vMSVhTyzuXzksf3LL/TG5CbW7wn7MDlG/TpAAthGojUMYYxzToIVCBV4XUYPrs9v4EQ0nBIpus61ythb/SKQD3vJBy0+5ynJwSYeaGeQxwQIDMpdhf4gSgB0mrmHZPRVNnpf2uH/gHYISUKI+IoS6H0+OjhvnvwCQPQtmdS/PAUu/8haU4p6CBQN90oywpnSHrdqJY0U0= ceph-d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:03:46.666 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.821+0000 7f63721a5700 1 Processor -- start 2024-04-25T13:03:46.666 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.821+0000 7f63721a5700 1 -- start start 2024-04-25T13:03:46.666 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.821+0000 7f63721a5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:46.666 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.821+0000 7f63721a5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f636c109060 con 0x7f636c108670 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.821+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.821+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53048/0 (socket says 172.21.15.72:53048) 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.821+0000 7f636b7fe700 1 -- 172.21.15.72:0/644497799 learned_addr learned my addr 172.21.15.72:0/644497799 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.821+0000 7f636b7fe700 1 -- 172.21.15.72:0/644497799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c109820 con 0x7f636c108670 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.821+0000 7f636b7fe700 1 --2- 172.21.15.72:0/644497799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c108a90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f635c0098a0 tx=0x7f635c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deae65ba9c7b8df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f636a7fc700 1 -- 172.21.15.72:0/644497799 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f635c004160 con 0x7f636c108670 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f636a7fc700 1 -- 172.21.15.72:0/644497799 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f635c004300 con 0x7f636c108670 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f636a7fc700 1 -- 172.21.15.72:0/644497799 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f635c011af0 con 0x7f636c108670 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f63721a5700 1 -- 172.21.15.72:0/644497799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 msgr2=0x7f636c108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f63721a5700 1 --2- 172.21.15.72:0/644497799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c108a90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f635c0098a0 tx=0x7f635c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f63721a5700 1 -- 172.21.15.72:0/644497799 shutdown_connections 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f63721a5700 1 --2- 172.21.15.72:0/644497799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c108a90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:46.667 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f63721a5700 1 -- 172.21.15.72:0/644497799 >> 172.21.15.72:0/644497799 conn(0x7f636c103c30 msgr2=0x7f636c106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f63721a5700 1 -- 172.21.15.72:0/644497799 shutdown_connections 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.822+0000 7f63721a5700 1 -- 172.21.15.72:0/644497799 wait complete. 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.823+0000 7f63721a5700 1 Processor -- start 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.823+0000 7f63721a5700 1 -- start start 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.823+0000 7f63721a5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.823+0000 7f63721a5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f636c10ca50 con 0x7f636c108670 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.823+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.823+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53054/0 (socket says 172.21.15.72:53054) 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.823+0000 7f636b7fe700 1 -- 172.21.15.72:0/525093213 learned_addr learned my addr 172.21.15.72:0/525093213 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.823+0000 7f636b7fe700 1 -- 172.21.15.72:0/525093213 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c19ca90 con 0x7f636c108670 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.824+0000 7f636b7fe700 1 --2- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c19c550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f635c009fb0 tx=0x7f635c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.824+0000 7f6368ff9700 1 -- 172.21.15.72:0/525093213 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f635c0036a0 con 0x7f636c108670 2024-04-25T13:03:46.668 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.824+0000 7f6368ff9700 1 -- 172.21.15.72:0/525093213 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f635c01a430 con 0x7f636c108670 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.824+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c19cd20 con 0x7f636c108670 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.824+0000 7f6368ff9700 1 -- 172.21.15.72:0/525093213 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f635c019550 con 0x7f636c108670 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.824+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c19fa10 con 0x7f636c108670 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.825+0000 7f6368ff9700 1 -- 172.21.15.72:0/525093213 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f635c019800 con 0x7f636c108670 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.825+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c10cbd0 con 0x7f636c108670 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.825+0000 7f6368ff9700 1 --2- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f63540395b0 0x7f635403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.825+0000 7f6368ff9700 1 -- 172.21.15.72:0/525093213 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f635c04da80 con 0x7f636c108670 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.825+0000 7f636affd700 1 --2- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f63540395b0 0x7f635403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.825+0000 7f636affd700 1 --2- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f63540395b0 0x7f635403ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:45.830+0000 7f6368ff9700 1 -- 172.21.15.72:0/525093213 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f635c00fda0 con 0x7f636c108670 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.027+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f636c104e80 con 0x7f63540395b0 2024-04-25T13:03:46.669 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.028+0000 7f6368ff9700 1 -- 172.21.15.72:0/525093213 <== mgr.14120 v2:172.21.15.72:6800/2774697366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f636c104e80 con 0x7f63540395b0 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f63540395b0 msgr2=0x7f635403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 --2- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f63540395b0 0x7f635403ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 msgr2=0x7f636c19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 --2- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c19c550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f635c009fb0 tx=0x7f635c011940 comp rx=0 tx=0).stop 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 shutdown_connections 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 --2- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f63540395b0 0x7f635403ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 --2- 172.21.15.72:0/525093213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636c108670 0x7f636c19c550 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 >> 172.21.15.72:0/525093213 conn(0x7f636c103c30 msgr2=0x7f636c104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 shutdown_connections 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:46.030+0000 7f63721a5700 1 -- 172.21.15.72:0/525093213 wait complete. 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T13:03:46.670 INFO:teuthology.orchestra.run.smithi072.stdout:Adding host smithi072... 2024-04-25T13:03:47.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:47 smithi072 ceph-mon[29199]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:48.135 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:48 smithi072 ceph-mon[29199]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi072", "addr": "172.21.15.72", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:48.136 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:48 smithi072 ceph-mon[29199]: Deploying cephadm binary to smithi072 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Added host 'smithi072' with addr '172.21.15.72' 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.088+0000 7fcc18195700 1 Processor -- start 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.088+0000 7fcc18195700 1 -- start start 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.088+0000 7fcc18195700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1010b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.088+0000 7fcc18195700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1007ce50 con 0x7fcc101090e0 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.088+0000 7fcc15f31700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1010b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.088+0000 7fcc15f31700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1010b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53068/0 (socket says 172.21.15.72:53068) 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.088+0000 7fcc15f31700 1 -- 172.21.15.72:0/1558333333 learned_addr learned my addr 172.21.15.72:0/1558333333 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc15f31700 1 -- 172.21.15.72:0/1558333333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1010ba10 con 0x7fcc101090e0 2024-04-25T13:03:51.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc15f31700 1 --2- 172.21.15.72:0/1558333333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1010b4d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098a0 tx=0x7fcc00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6bf4ef3f8bbb90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc14f2f700 1 -- 172.21.15.72:0/1558333333 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00004160 con 0x7fcc101090e0 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc14f2f700 1 -- 172.21.15.72:0/1558333333 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc00004300 con 0x7fcc101090e0 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc14f2f700 1 -- 172.21.15.72:0/1558333333 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00011af0 con 0x7fcc101090e0 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc18195700 1 -- 172.21.15.72:0/1558333333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 msgr2=0x7fcc1010b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc18195700 1 --2- 172.21.15.72:0/1558333333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1010b4d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098a0 tx=0x7fcc00009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc18195700 1 -- 172.21.15.72:0/1558333333 shutdown_connections 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc18195700 1 --2- 172.21.15.72:0/1558333333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1010b4d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.089+0000 7fcc18195700 1 -- 172.21.15.72:0/1558333333 >> 172.21.15.72:0/1558333333 conn(0x7fcc10102ad0 msgr2=0x7fcc10104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.090+0000 7fcc18195700 1 -- 172.21.15.72:0/1558333333 shutdown_connections 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.090+0000 7fcc18195700 1 -- 172.21.15.72:0/1558333333 wait complete. 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.090+0000 7fcc18195700 1 Processor -- start 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.090+0000 7fcc18195700 1 -- start start 2024-04-25T13:03:51.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.090+0000 7fcc18195700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.090+0000 7fcc18195700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1010c910 con 0x7fcc101090e0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.090+0000 7fcc15f31700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.091+0000 7fcc15f31700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53074/0 (socket says 172.21.15.72:53074) 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.091+0000 7fcc15f31700 1 -- 172.21.15.72:0/1840536224 learned_addr learned my addr 172.21.15.72:0/1840536224 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.091+0000 7fcc15f31700 1 -- 172.21.15.72:0/1840536224 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1019ca30 con 0x7fcc101090e0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.091+0000 7fcc15f31700 1 --2- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1019c4f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009fb0 tx=0x7fcc00011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.091+0000 7fcc06ffd700 1 -- 172.21.15.72:0/1840536224 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc000036a0 con 0x7fcc101090e0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.091+0000 7fcc06ffd700 1 -- 172.21.15.72:0/1840536224 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc0001a430 con 0x7fcc101090e0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.091+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1019ccc0 con 0x7fcc101090e0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.091+0000 7fcc06ffd700 1 -- 172.21.15.72:0/1840536224 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00019550 con 0x7fcc101090e0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.091+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1019f9b0 con 0x7fcc101090e0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.092+0000 7fcc06ffd700 1 -- 172.21.15.72:0/1840536224 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcc0001a8e0 con 0x7fcc101090e0 2024-04-25T13:03:51.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.092+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc100640a0 con 0x7fcc101090e0 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.092+0000 7fcc06ffd700 1 --2- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fcbfc039560 0x7fcbfc03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.092+0000 7fcc06ffd700 1 -- 172.21.15.72:0/1840536224 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc0004d980 con 0x7fcc101090e0 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.092+0000 7fcc15730700 1 --2- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fcbfc039560 0x7fcbfc03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.093+0000 7fcc15730700 1 --2- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fcbfc039560 0x7fcbfc03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c006fd0 tx=0x7fcc0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.097+0000 7fcc06ffd700 1 -- 172.21.15.72:0/1840536224 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc0000b3e0 con 0x7fcc101090e0 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:47.301+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi072", "addr": "172.21.15.72", "target": ["mon-mgr", ""]}) -- 0x7fcc10103d70 con 0x7fcbfc039560 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.366+0000 7fcc06ffd700 1 -- 172.21.15.72:0/1840536224 <== mgr.14120 v2:172.21.15.72:6800/2774697366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fcc10103d70 con 0x7fcbfc039560 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fcbfc039560 msgr2=0x7fcbfc03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 --2- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fcbfc039560 0x7fcbfc03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c006fd0 tx=0x7fcc0c006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 msgr2=0x7fcc1019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 --2- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1019c4f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009fb0 tx=0x7fcc00011940 comp rx=0 tx=0).stop 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 shutdown_connections 2024-04-25T13:03:51.980 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 --2- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fcbfc039560 0x7fcbfc03ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:51.981 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 --2- 172.21.15.72:0/1840536224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc101090e0 0x7fcc1019c4f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:51.981 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 >> 172.21.15.72:0/1840536224 conn(0x7fcc10102ad0 msgr2=0x7fcc10103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:51.981 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 shutdown_connections 2024-04-25T13:03:51.981 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:51.368+0000 7fcc18195700 1 -- 172.21.15.72:0/1840536224 wait complete. 2024-04-25T13:03:51.981 INFO:teuthology.orchestra.run.smithi072.stdout:Deploying unmanaged mon service... 2024-04-25T13:03:52.620 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:52 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:52.620 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:52 smithi072 ceph-mon[29199]: Added host smithi072 2024-04-25T13:03:52.620 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:52 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:53.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T13:03:53.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.389+0000 7f7f4553e700 1 Processor -- start 2024-04-25T13:03:53.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.389+0000 7f7f4553e700 1 -- start start 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.389+0000 7f7f4553e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f40108f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.389+0000 7f7f4553e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401094f0 con 0x7f7f40108b00 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.389+0000 7f7f3effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f40108f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.389+0000 7f7f3effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f40108f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53084/0 (socket says 172.21.15.72:53084) 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.389+0000 7f7f3effd700 1 -- 172.21.15.72:0/1998487180 learned_addr learned my addr 172.21.15.72:0/1998487180 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.389+0000 7f7f3effd700 1 -- 172.21.15.72:0/1998487180 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f40107760 con 0x7f7f40108b00 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f3effd700 1 --2- 172.21.15.72:0/1998487180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f40108f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f300098a0 tx=0x7f7f30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70039d363aa687b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f3dffb700 1 -- 172.21.15.72:0/1998487180 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f30004160 con 0x7f7f40108b00 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f3dffb700 1 -- 172.21.15.72:0/1998487180 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7f30004300 con 0x7f7f40108b00 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f3dffb700 1 -- 172.21.15.72:0/1998487180 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f30011af0 con 0x7f7f40108b00 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f4553e700 1 -- 172.21.15.72:0/1998487180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 msgr2=0x7f7f40108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f4553e700 1 --2- 172.21.15.72:0/1998487180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f40108f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f300098a0 tx=0x7f7f30009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:53.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f4553e700 1 -- 172.21.15.72:0/1998487180 shutdown_connections 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f4553e700 1 --2- 172.21.15.72:0/1998487180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f40108f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f4553e700 1 -- 172.21.15.72:0/1998487180 >> 172.21.15.72:0/1998487180 conn(0x7f7f401022b0 msgr2=0x7f7f401046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f4553e700 1 -- 172.21.15.72:0/1998487180 shutdown_connections 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.390+0000 7f7f4553e700 1 -- 172.21.15.72:0/1998487180 wait complete. 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.391+0000 7f7f4553e700 1 Processor -- start 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.391+0000 7f7f4553e700 1 -- start start 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.391+0000 7f7f4553e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f401070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.391+0000 7f7f4553e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4010b9a0 con 0x7f7f40108b00 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.391+0000 7f7f3effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f401070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.391+0000 7f7f3effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f401070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53090/0 (socket says 172.21.15.72:53090) 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.391+0000 7f7f3effd700 1 -- 172.21.15.72:0/2651894964 learned_addr learned my addr 172.21.15.72:0/2651894964 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.392+0000 7f7f3effd700 1 -- 172.21.15.72:0/2651894964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4019f2b0 con 0x7f7f40108b00 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.392+0000 7f7f3effd700 1 --2- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f401070f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009fb0 tx=0x7f7f30011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:53.381 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.392+0000 7f7f27fff700 1 -- 172.21.15.72:0/2651894964 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f300036a0 con 0x7f7f40108b00 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.392+0000 7f7f27fff700 1 -- 172.21.15.72:0/2651894964 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7f3001a430 con 0x7f7f40108b00 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.392+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4019f480 con 0x7f7f40108b00 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.392+0000 7f7f27fff700 1 -- 172.21.15.72:0/2651894964 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f30019550 con 0x7f7f40108b00 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.392+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4019f8a0 con 0x7f7f40108b00 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.393+0000 7f7f27fff700 1 -- 172.21.15.72:0/2651894964 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7f30019800 con 0x7f7f40108b00 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.393+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f400640a0 con 0x7f7f40108b00 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.393+0000 7f7f27fff700 1 --2- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f7f280395b0 0x7f7f2803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.393+0000 7f7f27fff700 1 -- 172.21.15.72:0/2651894964 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7f3004da30 con 0x7f7f40108b00 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.393+0000 7f7f3e7fc700 1 --2- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f7f280395b0 0x7f7f2803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.393+0000 7f7f3e7fc700 1 --2- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f7f280395b0 0x7f7f2803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7f34006fd0 tx=0x7f7f34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.398+0000 7f7f27fff700 1 -- 172.21.15.72:0/2651894964 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f3001a5e0 con 0x7f7f40108b00 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.621+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7f40103550 con 0x7f7f280395b0 2024-04-25T13:03:53.382 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.628+0000 7f7f27fff700 1 -- 172.21.15.72:0/2651894964 <== mgr.14120 v2:172.21.15.72:6800/2774697366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7f40103550 con 0x7f7f280395b0 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f7f280395b0 msgr2=0x7f7f2803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 --2- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f7f280395b0 0x7f7f2803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7f34006fd0 tx=0x7f7f34006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 msgr2=0x7f7f401070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 --2- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f401070f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009fb0 tx=0x7f7f30011940 comp rx=0 tx=0).stop 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 shutdown_connections 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 --2- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f7f280395b0 0x7f7f2803ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 --2- 172.21.15.72:0/2651894964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f40108b00 0x7f7f401070f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 >> 172.21.15.72:0/2651894964 conn(0x7f7f401022b0 msgr2=0x7f7f40102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 shutdown_connections 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:52.631+0000 7f7f4553e700 1 -- 172.21.15.72:0/2651894964 wait complete. 2024-04-25T13:03:53.383 INFO:teuthology.orchestra.run.smithi072.stdout:Deploying unmanaged mgr service... 2024-04-25T13:03:53.929 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:53 smithi072 ceph-mon[29199]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:53.929 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:53 smithi072 ceph-mon[29199]: Saving service mon spec with placement count:5 2024-04-25T13:03:53.929 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:53 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:55.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:55 smithi072 ceph-mon[29199]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:55.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:55 smithi072 ceph-mon[29199]: Saving service mgr spec with placement count:2 2024-04-25T13:03:55.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:55 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:55.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.816+0000 7f25d9d88700 1 Processor -- start 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.816+0000 7f25d9d88700 1 -- start start 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.816+0000 7f25d9d88700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d4108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.816+0000 7f25d9d88700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f25d4109000 con 0x7f25d4108610 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.816+0000 7f25d37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d4108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.816+0000 7f25d37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d4108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53098/0 (socket says 172.21.15.72:53098) 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.816+0000 7f25d37fe700 1 -- 172.21.15.72:0/4239205514 learned_addr learned my addr 172.21.15.72:0/4239205514 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.816+0000 7f25d37fe700 1 -- 172.21.15.72:0/4239205514 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d4109810 con 0x7f25d4108610 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.816+0000 7f25d37fe700 1 --2- 172.21.15.72:0/4239205514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d4108a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f25c80098a0 tx=0x7f25c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe429f0b96b3780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d27fc700 1 -- 172.21.15.72:0/4239205514 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25c8004160 con 0x7f25d4108610 2024-04-25T13:03:55.482 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d27fc700 1 -- 172.21.15.72:0/4239205514 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f25c8004300 con 0x7f25d4108610 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d27fc700 1 -- 172.21.15.72:0/4239205514 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25c8011af0 con 0x7f25d4108610 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d9d88700 1 -- 172.21.15.72:0/4239205514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 msgr2=0x7f25d4108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d9d88700 1 --2- 172.21.15.72:0/4239205514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d4108a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f25c80098a0 tx=0x7f25c8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d9d88700 1 -- 172.21.15.72:0/4239205514 shutdown_connections 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d9d88700 1 --2- 172.21.15.72:0/4239205514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d4108a30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d9d88700 1 -- 172.21.15.72:0/4239205514 >> 172.21.15.72:0/4239205514 conn(0x7f25d4103bf0 msgr2=0x7f25d4106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d9d88700 1 -- 172.21.15.72:0/4239205514 shutdown_connections 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.817+0000 7f25d9d88700 1 -- 172.21.15.72:0/4239205514 wait complete. 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.818+0000 7f25d9d88700 1 Processor -- start 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.818+0000 7f25d9d88700 1 -- start start 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.818+0000 7f25d9d88700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.818+0000 7f25d9d88700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f25d410ca40 con 0x7f25d4108610 2024-04-25T13:03:55.483 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.818+0000 7f25d37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.818+0000 7f25d37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d419c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53106/0 (socket says 172.21.15.72:53106) 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.818+0000 7f25d37fe700 1 -- 172.21.15.72:0/2126256798 learned_addr learned my addr 172.21.15.72:0/2126256798 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.819+0000 7f25d37fe700 1 -- 172.21.15.72:0/2126256798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d419cab0 con 0x7f25d4108610 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.819+0000 7f25d37fe700 1 --2- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d419c570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009fb0 tx=0x7f25c8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.819+0000 7f25d0ff9700 1 -- 172.21.15.72:0/2126256798 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25c80036a0 con 0x7f25d4108610 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.819+0000 7f25d0ff9700 1 -- 172.21.15.72:0/2126256798 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f25c801a430 con 0x7f25d4108610 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.819+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d419cd40 con 0x7f25d4108610 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.819+0000 7f25d0ff9700 1 -- 172.21.15.72:0/2126256798 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25c8019550 con 0x7f25d4108610 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.819+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d419fa30 con 0x7f25d4108610 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.820+0000 7f25d0ff9700 1 -- 172.21.15.72:0/2126256798 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f25c8019850 con 0x7f25d4108610 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.820+0000 7f25d0ff9700 1 --2- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f25bc039560 0x7f25bc03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.820+0000 7f25d0ff9700 1 -- 172.21.15.72:0/2126256798 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f25c804d940 con 0x7f25d4108610 2024-04-25T13:03:55.484 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.820+0000 7f25d2ffd700 1 --2- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f25bc039560 0x7f25bc03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.820+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d410cbc0 con 0x7f25d4108610 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.820+0000 7f25d2ffd700 1 --2- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f25bc039560 0x7f25bc03ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f25c4006fd0 tx=0x7f25c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:53.825+0000 7f25d0ff9700 1 -- 172.21.15.72:0/2126256798 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25c8023c90 con 0x7f25d4108610 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.040+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f25d4104e50 con 0x7f25bc039560 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.047+0000 7f25d0ff9700 1 -- 172.21.15.72:0/2126256798 <== mgr.14120 v2:172.21.15.72:6800/2774697366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f25d4104e50 con 0x7f25bc039560 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f25bc039560 msgr2=0x7f25bc03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 --2- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f25bc039560 0x7f25bc03ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f25c4006fd0 tx=0x7f25c4006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 msgr2=0x7f25d419c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 --2- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d419c570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009fb0 tx=0x7f25c8011940 comp rx=0 tx=0).stop 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 shutdown_connections 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 --2- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f25bc039560 0x7f25bc03ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 --2- 172.21.15.72:0/2126256798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25d4108610 0x7f25d419c570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:55.485 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 >> 172.21.15.72:0/2126256798 conn(0x7f25d4103bf0 msgr2=0x7f25d4104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:55.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 shutdown_connections 2024-04-25T13:03:55.486 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:54.049+0000 7f25d9d88700 1 -- 172.21.15.72:0/2126256798 wait complete. 2024-04-25T13:03:56.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:56 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:56.353 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:56 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:57.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.972+0000 7f2a976c8700 1 Processor -- start 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.972+0000 7f2a976c8700 1 -- start start 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.972+0000 7f2a976c8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a90107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.972+0000 7f2a976c8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2a90107f00 con 0x7f2a90107510 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.972+0000 7f2a95464700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a90107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.972+0000 7f2a95464700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a90107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40432/0 (socket says 172.21.15.72:40432) 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.972+0000 7f2a95464700 1 -- 172.21.15.72:0/39500010 learned_addr learned my addr 172.21.15.72:0/39500010 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.972+0000 7f2a95464700 1 -- 172.21.15.72:0/39500010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a90108710 con 0x7f2a90107510 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.973+0000 7f2a95464700 1 --2- 172.21.15.72:0/39500010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a90107930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0098a0 tx=0x7f2a8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ae1b57d067403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.973+0000 7f2a87fff700 1 -- 172.21.15.72:0/39500010 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a8c004160 con 0x7f2a90107510 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.973+0000 7f2a87fff700 1 -- 172.21.15.72:0/39500010 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a8c004300 con 0x7f2a90107510 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.973+0000 7f2a87fff700 1 -- 172.21.15.72:0/39500010 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a8c011a20 con 0x7f2a90107510 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 -- 172.21.15.72:0/39500010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 msgr2=0x7f2a90107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:57.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 --2- 172.21.15.72:0/39500010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a90107930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0098a0 tx=0x7f2a8c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 -- 172.21.15.72:0/39500010 shutdown_connections 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 --2- 172.21.15.72:0/39500010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a90107930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 -- 172.21.15.72:0/39500010 >> 172.21.15.72:0/39500010 conn(0x7f2a90102ad0 msgr2=0x7f2a90104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 -- 172.21.15.72:0/39500010 shutdown_connections 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 -- 172.21.15.72:0/39500010 wait complete. 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 Processor -- start 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 -- start start 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.974+0000 7f2a976c8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a9019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a976c8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9010bac0 con 0x7f2a90107510 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a95464700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a9019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a95464700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a9019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40444/0 (socket says 172.21.15.72:40444) 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a95464700 1 -- 172.21.15.72:0/58307859 learned_addr learned my addr 172.21.15.72:0/58307859 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a95464700 1 -- 172.21.15.72:0/58307859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9019b950 con 0x7f2a90107510 2024-04-25T13:03:57.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a95464700 1 --2- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a9019b410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0119f0 tx=0x7f2a8c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a867fc700 1 -- 172.21.15.72:0/58307859 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a8c0039e0 con 0x7f2a90107510 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a867fc700 1 -- 172.21.15.72:0/58307859 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a8c01aa10 con 0x7f2a90107510 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9019bbe0 con 0x7f2a90107510 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a867fc700 1 -- 172.21.15.72:0/58307859 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a8c023930 con 0x7f2a90107510 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.975+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9007bfd0 con 0x7f2a90107510 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.976+0000 7f2a867fc700 1 -- 172.21.15.72:0/58307859 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2a8c023ba0 con 0x7f2a90107510 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.976+0000 7f2a867fc700 1 --2- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f2a7c0395b0 0x7f2a7c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.976+0000 7f2a867fc700 1 -- 172.21.15.72:0/58307859 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2a8c04f8a0 con 0x7f2a90107510 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.976+0000 7f2a94c63700 1 --2- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f2a7c0395b0 0x7f2a7c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.977+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9010bc40 con 0x7f2a90107510 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.977+0000 7f2a94c63700 1 --2- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f2a7c0395b0 0x7f2a7c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2a80006fd0 tx=0x7f2a80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:55.982+0000 7f2a867fc700 1 -- 172.21.15.72:0/58307859 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a8c02a2b0 con 0x7f2a90107510 2024-04-25T13:03:57.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.219+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f2a9007c4c0 con 0x7f2a90107510 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.222+0000 7f2a867fc700 1 -- 172.21.15.72:0/58307859 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f2a8c032020 con 0x7f2a90107510 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f2a7c0395b0 msgr2=0x7f2a7c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 --2- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f2a7c0395b0 0x7f2a7c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2a80006fd0 tx=0x7f2a80006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 msgr2=0x7f2a9019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 --2- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a9019b410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0119f0 tx=0x7f2a8c011ad0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 shutdown_connections 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 --2- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f2a7c0395b0 0x7f2a7c03ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 --2- 172.21.15.72:0/58307859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a90107510 0x7f2a9019b410 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 >> 172.21.15.72:0/58307859 conn(0x7f2a90102ad0 msgr2=0x7f2a901035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 shutdown_connections 2024-04-25T13:03:57.359 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:56.225+0000 7f2a976c8700 1 -- 172.21.15.72:0/58307859 wait complete. 2024-04-25T13:03:57.474 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:57 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/58307859' entity='client.admin' 2024-04-25T13:03:58.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.778+0000 7f3134821700 1 Processor -- start 2024-04-25T13:03:58.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.778+0000 7f3134821700 1 -- start start 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.778+0000 7f3134821700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313010b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.778+0000 7f3134821700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f313007ce70 con 0x7f31301090c0 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.778+0000 7f312e59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313010b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.778+0000 7f312e59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313010b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40446/0 (socket says 172.21.15.72:40446) 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.778+0000 7f312e59c700 1 -- 172.21.15.72:0/238485746 learned_addr learned my addr 172.21.15.72:0/238485746 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.778+0000 7f312e59c700 1 -- 172.21.15.72:0/238485746 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313010b9f0 con 0x7f31301090c0 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.779+0000 7f312e59c700 1 --2- 172.21.15.72:0/238485746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313010b4b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f31200098a0 tx=0x7f3120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1f2ba7085b7099 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.779+0000 7f312d59a700 1 -- 172.21.15.72:0/238485746 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3120004160 con 0x7f31301090c0 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.779+0000 7f312d59a700 1 -- 172.21.15.72:0/238485746 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3120004300 con 0x7f31301090c0 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.779+0000 7f312d59a700 1 -- 172.21.15.72:0/238485746 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3120011a20 con 0x7f31301090c0 2024-04-25T13:03:58.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.779+0000 7f3134821700 1 -- 172.21.15.72:0/238485746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 msgr2=0x7f313010b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.779+0000 7f3134821700 1 --2- 172.21.15.72:0/238485746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313010b4b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f31200098a0 tx=0x7f3120009bb0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.779+0000 7f3134821700 1 -- 172.21.15.72:0/238485746 shutdown_connections 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.779+0000 7f3134821700 1 --2- 172.21.15.72:0/238485746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313010b4b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.779+0000 7f3134821700 1 -- 172.21.15.72:0/238485746 >> 172.21.15.72:0/238485746 conn(0x7f3130102ad0 msgr2=0x7f3130104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.780+0000 7f3134821700 1 -- 172.21.15.72:0/238485746 shutdown_connections 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.780+0000 7f3134821700 1 -- 172.21.15.72:0/238485746 wait complete. 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.780+0000 7f3134821700 1 Processor -- start 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.780+0000 7f3134821700 1 -- start start 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f3134821700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f3134821700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f313010c9f0 con 0x7f31301090c0 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f312e59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f312e59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40460/0 (socket says 172.21.15.72:40460) 2024-04-25T13:03:58.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f312e59c700 1 -- 172.21.15.72:0/407264984 learned_addr learned my addr 172.21.15.72:0/407264984 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f312e59c700 1 -- 172.21.15.72:0/407264984 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313019c9e0 con 0x7f31301090c0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f312e59c700 1 --2- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313019c4a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f31200116e0 tx=0x7f3120011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f311f7fe700 1 -- 172.21.15.72:0/407264984 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3120003b20 con 0x7f31301090c0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f311f7fe700 1 -- 172.21.15.72:0/407264984 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f312001b8f0 con 0x7f31301090c0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313019cc70 con 0x7f31301090c0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f311f7fe700 1 -- 172.21.15.72:0/407264984 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31200238c0 con 0x7f31301090c0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.781+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313019f960 con 0x7f31301090c0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.782+0000 7f311f7fe700 1 -- 172.21.15.72:0/407264984 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3120023b70 con 0x7f31301090c0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.782+0000 7f311f7fe700 1 --2- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f3118039560 0x7f311803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.782+0000 7f311f7fe700 1 -- 172.21.15.72:0/407264984 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f312004e040 con 0x7f31301090c0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.782+0000 7f312dd9b700 1 --2- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f3118039560 0x7f311803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.782+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31300640a0 con 0x7f31301090c0 2024-04-25T13:03:58.622 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.783+0000 7f312dd9b700 1 --2- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f3118039560 0x7f311803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3124006fd0 tx=0x7f3124006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.787+0000 7f311f7fe700 1 -- 172.21.15.72:0/407264984 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f312001f030 con 0x7f31301090c0 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.994+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f313019fb30 con 0x7f31301090c0 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.997+0000 7f311f7fe700 1 -- 172.21.15.72:0/407264984 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3120020d70 con 0x7f31301090c0 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.999+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f3118039560 msgr2=0x7f311803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.999+0000 7f3134821700 1 --2- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f3118039560 0x7f311803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3124006fd0 tx=0x7f3124006e40 comp rx=0 tx=0).stop 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.999+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 msgr2=0x7f313019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.999+0000 7f3134821700 1 --2- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313019c4a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f31200116e0 tx=0x7f3120011aa0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.999+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 shutdown_connections 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:57.999+0000 7f3134821700 1 --2- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f3118039560 0x7f311803ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:58.000+0000 7f3134821700 1 --2- 172.21.15.72:0/407264984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31301090c0 0x7f313019c4a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:58.000+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 >> 172.21.15.72:0/407264984 conn(0x7f3130102ad0 msgr2=0x7f3130103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:58.000+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 shutdown_connections 2024-04-25T13:03:58.623 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:58.000+0000 7f3134821700 1 -- 172.21.15.72:0/407264984 wait complete. 2024-04-25T13:03:58.624 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling the dashboard module... 2024-04-25T13:03:59.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:58 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:03:59.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:03:58 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/407264984' entity='client.admin' 2024-04-25T13:04:00.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:00 smithi072 ceph-mon[29199]: from='mgr.14120 172.21.15.72:0/2869514866' entity='mgr.x' 2024-04-25T13:04:00.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:00 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3601860405' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T13:04:00.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:00 smithi072 ceph-mon[29199]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:00.523 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: ignoring --setuser ceph since I am not root 2024-04-25T13:04:00.523 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: ignoring --setgroup ceph since I am not root 2024-04-25T13:04:00.523 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:00.428+0000 7f562a864200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:04:00.994 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:00.610+0000 7f562a864200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:04:00.995 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:00.714+0000 7f562a864200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.064+0000 7fe00d9c7700 1 Processor -- start 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.064+0000 7fe00d9c7700 1 -- start start 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.065+0000 7fe00d9c7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe008106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.065+0000 7fe00d9c7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe008107310 con 0x7fe008106920 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.065+0000 7fe006ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe008106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.065+0000 7fe006ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe008106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40472/0 (socket says 172.21.15.72:40472) 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.065+0000 7fe006ffd700 1 -- 172.21.15.72:0/699020466 learned_addr learned my addr 172.21.15.72:0/699020466 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.065+0000 7fe006ffd700 1 -- 172.21.15.72:0/699020466 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe008107640 con 0x7fe008106920 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.065+0000 7fe006ffd700 1 --2- 172.21.15.72:0/699020466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe008106d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdff80098a0 tx=0x7fdff8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22a4430612137e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:01.004 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.065+0000 7fe005ffb700 1 -- 172.21.15.72:0/699020466 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff8004160 con 0x7fe008106920 2024-04-25T13:04:01.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.066+0000 7fe005ffb700 1 -- 172.21.15.72:0/699020466 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff8004300 con 0x7fe008106920 2024-04-25T13:04:01.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.066+0000 7fe005ffb700 1 -- 172.21.15.72:0/699020466 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff8011a20 con 0x7fe008106920 2024-04-25T13:04:01.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.066+0000 7fe00d9c7700 1 -- 172.21.15.72:0/699020466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 msgr2=0x7fe008106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:01.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.066+0000 7fe00d9c7700 1 --2- 172.21.15.72:0/699020466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe008106d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdff80098a0 tx=0x7fdff8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:04:01.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.066+0000 7fe00d9c7700 1 -- 172.21.15.72:0/699020466 shutdown_connections 2024-04-25T13:04:01.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.066+0000 7fe00d9c7700 1 --2- 172.21.15.72:0/699020466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe008106d40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:01.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.066+0000 7fe00d9c7700 1 -- 172.21.15.72:0/699020466 >> 172.21.15.72:0/699020466 conn(0x7fe0081022b0 msgr2=0x7fe0081046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:01.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.066+0000 7fe00d9c7700 1 -- 172.21.15.72:0/699020466 shutdown_connections 2024-04-25T13:04:01.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.066+0000 7fe00d9c7700 1 -- 172.21.15.72:0/699020466 wait complete. 2024-04-25T13:04:01.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.067+0000 7fe00d9c7700 1 Processor -- start 2024-04-25T13:04:01.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.067+0000 7fe00d9c7700 1 -- start start 2024-04-25T13:04:01.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.067+0000 7fe00d9c7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe00819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:01.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.067+0000 7fe00d9c7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe008108a00 con 0x7fe008106920 2024-04-25T13:04:01.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.067+0000 7fe006ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe00819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:01.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.067+0000 7fe006ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe00819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40478/0 (socket says 172.21.15.72:40478) 2024-04-25T13:04:01.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.067+0000 7fe006ffd700 1 -- 172.21.15.72:0/3601860405 learned_addr learned my addr 172.21.15.72:0/3601860405 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:01.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.068+0000 7fe006ffd700 1 -- 172.21.15.72:0/3601860405 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00819ec80 con 0x7fe008106920 2024-04-25T13:04:01.008 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.068+0000 7fe006ffd700 1 --2- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe00819e740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdff80119f0 tx=0x7fdff8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:01.008 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.068+0000 7fe00c9c5700 1 -- 172.21.15.72:0/3601860405 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff80039e0 con 0x7fe008106920 2024-04-25T13:04:01.008 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.068+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00819ef10 con 0x7fe008106920 2024-04-25T13:04:01.008 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.068+0000 7fe00c9c5700 1 -- 172.21.15.72:0/3601860405 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff801b8f0 con 0x7fe008106920 2024-04-25T13:04:01.008 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.068+0000 7fe00c9c5700 1 -- 172.21.15.72:0/3601860405 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff80237a0 con 0x7fe008106920 2024-04-25T13:04:01.008 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.068+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0081a1c00 con 0x7fe008106920 2024-04-25T13:04:01.008 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.069+0000 7fe00c9c5700 1 -- 172.21.15.72:0/3601860405 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdff801b440 con 0x7fe008106920 2024-04-25T13:04:01.008 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.069+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0080640a0 con 0x7fe008106920 2024-04-25T13:04:01.009 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.069+0000 7fe00c9c5700 1 --2- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fdff0039560 0x7fdff003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:01.009 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.069+0000 7fe00c9c5700 1 -- 172.21.15.72:0/3601860405 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdff804de70 con 0x7fe008106920 2024-04-25T13:04:01.009 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.070+0000 7fe0067fc700 1 --2- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fdff0039560 0x7fdff003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:01.009 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.070+0000 7fe0067fc700 1 --2- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fdff0039560 0x7fdff003ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdffc006fd0 tx=0x7fdffc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:01.009 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.079+0000 7fe00c9c5700 1 -- 172.21.15.72:0/3601860405 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdff801f030 con 0x7fe008106920 2024-04-25T13:04:01.009 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:03:59.355+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fe008197ca0 con 0x7fe008106920 2024-04-25T13:04:01.009 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.254+0000 7fe00c9c5700 1 -- 172.21.15.72:0/3601860405 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fe008197ca0 con 0x7fe008106920 2024-04-25T13:04:01.009 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.255+0000 7fe00c9c5700 1 -- 172.21.15.72:0/3601860405 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fdff802c3f0 con 0x7fe008106920 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.257+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fdff0039560 msgr2=0x7fdff003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.257+0000 7fe00d9c7700 1 --2- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fdff0039560 0x7fdff003ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdffc006fd0 tx=0x7fdffc006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.258+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 msgr2=0x7fe00819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.258+0000 7fe00d9c7700 1 --2- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe00819e740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdff80119f0 tx=0x7fdff8011ad0 comp rx=0 tx=0).stop 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.258+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 shutdown_connections 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.258+0000 7fe00d9c7700 1 --2- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7fdff0039560 0x7fdff003ba20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.258+0000 7fe00d9c7700 1 --2- 172.21.15.72:0/3601860405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe008106920 0x7fe00819e740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.258+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 >> 172.21.15.72:0/3601860405 conn(0x7fe0081022b0 msgr2=0x7fe008194d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.258+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 shutdown_connections 2024-04-25T13:04:01.010 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:00.258+0000 7fe00d9c7700 1 -- 172.21.15.72:0/3601860405 wait complete. 2024-04-25T13:04:01.249 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:01.023+0000 7f562a864200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:04:01.249 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:01.107+0000 7f562a864200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:04:01.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:01 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3601860405' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T13:04:01.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:01 smithi072 ceph-mon[29199]: mgrmap e9: x(active, since 20s) 2024-04-25T13:04:02.426 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:02 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2281368331' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:04:02.427 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:02.313+0000 7f562a864200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:04:02.699 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:02.486+0000 7f562a864200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:04:02.699 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:02.655+0000 7f562a864200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:04:02.707 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.462+0000 7f4f46ac2700 1 Processor -- start 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.462+0000 7f4f46ac2700 1 -- start start 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.462+0000 7f4f46ac2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f40107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.462+0000 7f4f46ac2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40107f00 con 0x7f4f40107510 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.463+0000 7f4f4485e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f40107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:02.708 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.463+0000 7f4f4485e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f40107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40490/0 (socket says 172.21.15.72:40490) 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.463+0000 7f4f4485e700 1 -- 172.21.15.72:0/2961109824 learned_addr learned my addr 172.21.15.72:0/2961109824 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.463+0000 7f4f4485e700 1 -- 172.21.15.72:0/2961109824 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f40108710 con 0x7f4f40107510 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.463+0000 7f4f4485e700 1 --2- 172.21.15.72:0/2961109824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f40107930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f340098a0 tx=0x7f4f34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46faa7f2c866f300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.463+0000 7f4f3f7fe700 1 -- 172.21.15.72:0/2961109824 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f34004160 con 0x7f4f40107510 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.463+0000 7f4f3f7fe700 1 -- 172.21.15.72:0/2961109824 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f34004300 con 0x7f4f40107510 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.463+0000 7f4f3f7fe700 1 -- 172.21.15.72:0/2961109824 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f34011a20 con 0x7f4f40107510 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.464+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2961109824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 msgr2=0x7f4f40107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.464+0000 7f4f46ac2700 1 --2- 172.21.15.72:0/2961109824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f40107930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f340098a0 tx=0x7f4f34009bb0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.464+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2961109824 shutdown_connections 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.464+0000 7f4f46ac2700 1 --2- 172.21.15.72:0/2961109824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f40107930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.464+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2961109824 >> 172.21.15.72:0/2961109824 conn(0x7f4f40102b10 msgr2=0x7f4f40104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.464+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2961109824 shutdown_connections 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.464+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2961109824 wait complete. 2024-04-25T13:04:02.709 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.464+0000 7f4f46ac2700 1 Processor -- start 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.464+0000 7f4f46ac2700 1 -- start start 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f46ac2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f401925f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f46ac2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010bac0 con 0x7f4f40107510 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f4485e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f401925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f4485e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f401925f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40492/0 (socket says 172.21.15.72:40492) 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f4485e700 1 -- 172.21.15.72:0/2281368331 learned_addr learned my addr 172.21.15.72:0/2281368331 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f4485e700 1 -- 172.21.15.72:0/2281368331 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f40192b30 con 0x7f4f40107510 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f4485e700 1 --2- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f401925f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f34000c00 tx=0x7f4f34011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f3dffb700 1 -- 172.21.15.72:0/2281368331 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f34003aa0 con 0x7f4f40107510 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f3dffb700 1 -- 172.21.15.72:0/2281368331 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3401b8f0 con 0x7f4f40107510 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f40190d20 con 0x7f4f40107510 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.465+0000 7f4f3dffb700 1 -- 172.21.15.72:0/2281368331 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f34023890 con 0x7f4f40107510 2024-04-25T13:04:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.466+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f40191260 con 0x7f4f40107510 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.466+0000 7f4f3dffb700 1 -- 172.21.15.72:0/2281368331 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f4f3402c3f0 con 0x7f4f40107510 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.466+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4010bc40 con 0x7f4f40107510 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.467+0000 7f4f3dffb700 1 --2- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f4f30039660 0x7f4f3003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.467+0000 7f4f3dffb700 1 -- 172.21.15.72:0/2281368331 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4f3404de80 con 0x7f4f40107510 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.467+0000 7f4f3ffff700 1 -- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f4f30039660 msgr2=0x7f4f3003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2774697366 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.467+0000 7f4f3ffff700 1 --2- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f4f30039660 0x7f4f3003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.472+0000 7f4f3dffb700 1 -- 172.21.15.72:0/2281368331 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f3401f030 con 0x7f4f40107510 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.668+0000 7f4f3ffff700 1 -- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f4f30039660 msgr2=0x7f4f3003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2774697366 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.668+0000 7f4f3ffff700 1 --2- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f4f30039660 0x7f4f3003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.775+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4f40196320 con 0x7f4f40107510 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.775+0000 7f4f3dffb700 1 -- 172.21.15.72:0/2281368331 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f4f3401bda0 con 0x7f4f40107510 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.777+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f4f30039660 msgr2=0x7f4f3003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:04:02.711 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.777+0000 7f4f46ac2700 1 --2- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f4f30039660 0x7f4f3003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.777+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 msgr2=0x7f4f401925f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.777+0000 7f4f46ac2700 1 --2- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f401925f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f34000c00 tx=0x7f4f34011aa0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.777+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 shutdown_connections 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.777+0000 7f4f46ac2700 1 --2- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7f4f30039660 0x7f4f3003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.777+0000 7f4f46ac2700 1 --2- 172.21.15.72:0/2281368331 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f40107510 0x7f4f401925f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.778+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 >> 172.21.15.72:0/2281368331 conn(0x7f4f40102b10 msgr2=0x7f4f401039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.778+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 shutdown_connections 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:01.778+0000 7f4f46ac2700 1 -- 172.21.15.72:0/2281368331 wait complete. 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for the mgr to restart... 2024-04-25T13:04:02.712 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr epoch 9... 2024-04-25T13:04:03.249 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:02.951+0000 7f562a864200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:04:03.249 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:03.043+0000 7f562a864200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:04:03.249 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:03.250+0000 7f562a864200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:04:03.522 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:03.443+0000 7f562a864200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:04:04.446 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:04.029+0000 7f562a864200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:04:04.740 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:04.446+0000 7f562a864200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:04:04.740 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:04.541+0000 7f562a864200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:04:04.740 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:04.741+0000 7f562a864200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:04:05.522 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:05.255+0000 7f562a864200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:04:05.523 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:05.347+0000 7f562a864200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:04:05.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:05.588+0000 7f562a864200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:04:05.931 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:05.720+0000 7f562a864200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:04:05.931 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:05.815+0000 7f562a864200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:04:05.931 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:05.931+0000 7f562a864200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:04:06.522 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:06.230+0000 7f562a864200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:04:07.097 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:06.826+0000 7f562a864200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:04:07.098 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:04:07.043+0000 7f562a864200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: Active manager daemon x restarted 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: Activating manager daemon x 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: mgrmap e10: x(active, starting, since 0.0031062s) 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: Manager daemon x is now available 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:04:08.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:04:08.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:08 smithi072 ceph-mon[29199]: mgrmap e11: x(active, since 1.00643s) 2024-04-25T13:04:08.990 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:04:08.990 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T13:04:08.990 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:04:08.990 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:04:08.990 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.144+0000 7ffb6722d700 1 Processor -- start 2024-04-25T13:04:08.990 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.144+0000 7ffb6722d700 1 -- start start 2024-04-25T13:04:08.990 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.144+0000 7ffb6722d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6010b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.144+0000 7ffb6722d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60063d40 con 0x7ffb601090d0 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.144+0000 7ffb64fc9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6010b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.144+0000 7ffb64fc9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6010b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40508/0 (socket says 172.21.15.72:40508) 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.144+0000 7ffb64fc9700 1 -- 172.21.15.72:0/1281161246 learned_addr learned my addr 172.21.15.72:0/1281161246 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.145+0000 7ffb64fc9700 1 -- 172.21.15.72:0/1281161246 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6010ba00 con 0x7ffb601090d0 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.145+0000 7ffb64fc9700 1 --2- 172.21.15.72:0/1281161246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6010b4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffb540098a0 tx=0x7ffb54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16237d408f96904 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.145+0000 7ffb5f7fe700 1 -- 172.21.15.72:0/1281161246 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54004160 con 0x7ffb601090d0 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.145+0000 7ffb5f7fe700 1 -- 172.21.15.72:0/1281161246 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb54004300 con 0x7ffb601090d0 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.145+0000 7ffb5f7fe700 1 -- 172.21.15.72:0/1281161246 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54011a20 con 0x7ffb601090d0 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.146+0000 7ffb6722d700 1 -- 172.21.15.72:0/1281161246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 msgr2=0x7ffb6010b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.146+0000 7ffb6722d700 1 --2- 172.21.15.72:0/1281161246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6010b4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffb540098a0 tx=0x7ffb54009bb0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.146+0000 7ffb6722d700 1 -- 172.21.15.72:0/1281161246 shutdown_connections 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.146+0000 7ffb6722d700 1 --2- 172.21.15.72:0/1281161246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6010b4c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.146+0000 7ffb6722d700 1 -- 172.21.15.72:0/1281161246 >> 172.21.15.72:0/1281161246 conn(0x7ffb60102b80 msgr2=0x7ffb60104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.146+0000 7ffb6722d700 1 -- 172.21.15.72:0/1281161246 shutdown_connections 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.146+0000 7ffb6722d700 1 -- 172.21.15.72:0/1281161246 wait complete. 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb6722d700 1 Processor -- start 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb6722d700 1 -- start start 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb6722d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb6722d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601079f0 con 0x7ffb601090d0 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb64fc9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb64fc9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40520/0 (socket says 172.21.15.72:40520) 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb64fc9700 1 -- 172.21.15.72:0/670046910 learned_addr learned my addr 172.21.15.72:0/670046910 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb64fc9700 1 -- 172.21.15.72:0/670046910 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6019a7a0 con 0x7ffb601090d0 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb64fc9700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6019a260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb54004660 tx=0x7ffb54011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54003aa0 con 0x7ffb601090d0 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb5401b8f0 con 0x7ffb601090d0 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.147+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54023900 con 0x7ffb601090d0 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.148+0000 7ffb6722d700 1 -- 172.21.15.72:0/670046910 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6019aa90 con 0x7ffb601090d0 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.148+0000 7ffb6722d700 1 -- 172.21.15.72:0/670046910 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6019d720 con 0x7ffb601090d0 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.149+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ffb54023b80 con 0x7ffb601090d0 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.149+0000 7ffb5dffb700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 0x7ffb5003ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.149+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 --> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffb50040630 con 0x7ffb5003da60 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.149+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffb5404dea0 con 0x7ffb601090d0 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.149+0000 7ffb5ffff700 1 -- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 msgr2=0x7ffb5003ff20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2774697366 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.149+0000 7ffb5ffff700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 0x7ffb5003ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.350+0000 7ffb5ffff700 1 -- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 msgr2=0x7ffb5003ff20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2774697366 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.350+0000 7ffb5ffff700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 0x7ffb5003ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.751+0000 7ffb5ffff700 1 -- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 msgr2=0x7ffb5003ff20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2774697366 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:03.751+0000 7ffb5ffff700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 0x7ffb5003ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:04.552+0000 7ffb5ffff700 1 -- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 msgr2=0x7ffb5003ff20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2774697366 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:04.552+0000 7ffb5ffff700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 0x7ffb5003ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:06.153+0000 7ffb5ffff700 1 -- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 msgr2=0x7ffb5003ff20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2774697366 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:06.153+0000 7ffb5ffff700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 0x7ffb5003ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:07.046+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7ffb5401baa0 con 0x7ffb601090d0 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:07.046+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 msgr2=0x7ffb5003ff20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:07.046+0000 7ffb5dffb700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 0x7ffb5003ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.048+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7ffb5402c910 con 0x7ffb601090d0 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.049+0000 7ffb5dffb700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb50041530 0x7ffb50043920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.049+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffb50040630 con 0x7ffb50041530 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.049+0000 7ffb5ffff700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb50041530 0x7ffb50043920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.050+0000 7ffb5ffff700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb50041530 0x7ffb50043920 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c003a10 tx=0x7ffb4c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.050+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ffb50040630 con 0x7ffb50041530 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.057+0000 7ffb6722d700 1 -- 172.21.15.72:0/670046910 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ffb600640a0 con 0x7ffb50041530 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.057+0000 7ffb5dffb700 1 -- 172.21.15.72:0/670046910 <== mgr.14152 v2:172.21.15.72:6800/3348457169 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ffb600640a0 con 0x7ffb50041530 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 -- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb50041530 msgr2=0x7ffb50043920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb50041530 0x7ffb50043920 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c003a10 tx=0x7ffb4c0092b0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 -- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 msgr2=0x7ffb6019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6019a260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb54004660 tx=0x7ffb54011aa0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.995 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 -- 172.21.15.72:0/670046910 shutdown_connections 2024-04-25T13:04:08.996 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb50041530 0x7ffb50043920 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.996 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb601090d0 0x7ffb6019a260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.996 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 --2- 172.21.15.72:0/670046910 >> [v2:172.21.15.72:6800/2774697366,v1:172.21.15.72:6801/2774697366] conn(0x7ffb5003da60 0x7ffb5003ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.996 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 -- 172.21.15.72:0/670046910 >> 172.21.15.72:0/670046910 conn(0x7ffb60102b80 msgr2=0x7ffb60190be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:08.996 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 -- 172.21.15.72:0/670046910 shutdown_connections 2024-04-25T13:04:08.996 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:08.058+0000 7ffb6722d700 1 -- 172.21.15.72:0/670046910 wait complete. 2024-04-25T13:04:08.996 INFO:teuthology.orchestra.run.smithi072.stdout:mgr epoch 9 is available 2024-04-25T13:04:08.996 INFO:teuthology.orchestra.run.smithi072.stdout:Generating a dashboard self-signed certificate... 2024-04-25T13:04:09.231 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:09 smithi072 ceph-mon[29199]: [25/Apr/2024:13:04:08] ENGINE Bus STARTING 2024-04-25T13:04:09.231 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:09 smithi072 ceph-mon[29199]: [25/Apr/2024:13:04:08] ENGINE Serving on https://172.21.15.72:7150 2024-04-25T13:04:09.231 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:09 smithi072 ceph-mon[29199]: [25/Apr/2024:13:04:08] ENGINE Serving on http://172.21.15.72:8765 2024-04-25T13:04:09.231 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:09 smithi072 ceph-mon[29199]: [25/Apr/2024:13:04:08] ENGINE Bus STARTED 2024-04-25T13:04:09.231 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:10.321 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T13:04:10.322 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.430+0000 7ff54e134700 1 Processor -- start 2024-04-25T13:04:10.322 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.430+0000 7ff54e134700 1 -- start start 2024-04-25T13:04:10.324 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff54e134700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff548108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.324 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff54e134700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff548109000 con 0x7ff548108610 2024-04-25T13:04:10.324 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff5477fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff548108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.324 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff5477fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff548108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33360/0 (socket says 172.21.15.72:33360) 2024-04-25T13:04:10.324 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff5477fe700 1 -- 172.21.15.72:0/4205254670 learned_addr learned my addr 172.21.15.72:0/4205254670 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:10.324 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff5477fe700 1 -- 172.21.15.72:0/4205254670 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff548109810 con 0x7ff548108610 2024-04-25T13:04:10.324 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff5477fe700 1 --2- 172.21.15.72:0/4205254670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff548108a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5380098a0 tx=0x7ff538009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c314ce8609c99df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:10.324 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff5467fc700 1 -- 172.21.15.72:0/4205254670 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff538004160 con 0x7ff548108610 2024-04-25T13:04:10.325 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff5467fc700 1 -- 172.21.15.72:0/4205254670 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff538004300 con 0x7ff548108610 2024-04-25T13:04:10.325 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.431+0000 7ff5467fc700 1 -- 172.21.15.72:0/4205254670 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff538011a20 con 0x7ff548108610 2024-04-25T13:04:10.325 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.432+0000 7ff54e134700 1 -- 172.21.15.72:0/4205254670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 msgr2=0x7ff548108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.325 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.432+0000 7ff54e134700 1 --2- 172.21.15.72:0/4205254670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff548108a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5380098a0 tx=0x7ff538009bb0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.325 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.432+0000 7ff54e134700 1 -- 172.21.15.72:0/4205254670 shutdown_connections 2024-04-25T13:04:10.325 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.432+0000 7ff54e134700 1 --2- 172.21.15.72:0/4205254670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff548108a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.329 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.432+0000 7ff54e134700 1 -- 172.21.15.72:0/4205254670 >> 172.21.15.72:0/4205254670 conn(0x7ff548103c10 msgr2=0x7ff548106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:10.329 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.432+0000 7ff54e134700 1 -- 172.21.15.72:0/4205254670 shutdown_connections 2024-04-25T13:04:10.329 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.432+0000 7ff54e134700 1 -- 172.21.15.72:0/4205254670 wait complete. 2024-04-25T13:04:10.329 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.433+0000 7ff54e134700 1 Processor -- start 2024-04-25T13:04:10.329 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.433+0000 7ff54e134700 1 -- start start 2024-04-25T13:04:10.329 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.433+0000 7ff54e134700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff54819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.329 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.433+0000 7ff54e134700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff54810cbc0 con 0x7ff548108610 2024-04-25T13:04:10.329 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.433+0000 7ff5477fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff54819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.329 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.433+0000 7ff5477fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff54819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33374/0 (socket says 172.21.15.72:33374) 2024-04-25T13:04:10.330 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.433+0000 7ff5477fe700 1 -- 172.21.15.72:0/2853267606 learned_addr learned my addr 172.21.15.72:0/2853267606 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:10.330 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.433+0000 7ff5477fe700 1 -- 172.21.15.72:0/2853267606 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54819ca70 con 0x7ff548108610 2024-04-25T13:04:10.330 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.433+0000 7ff5477fe700 1 --2- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff54819c530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5380119f0 tx=0x7ff538011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:10.330 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.434+0000 7ff544ff9700 1 -- 172.21.15.72:0/2853267606 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5380039e0 con 0x7ff548108610 2024-04-25T13:04:10.330 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.434+0000 7ff544ff9700 1 -- 172.21.15.72:0/2853267606 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53801b8f0 con 0x7ff548108610 2024-04-25T13:04:10.330 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.434+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54819cd00 con 0x7ff548108610 2024-04-25T13:04:10.330 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.434+0000 7ff544ff9700 1 -- 172.21.15.72:0/2853267606 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5380237a0 con 0x7ff548108610 2024-04-25T13:04:10.330 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.434+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54819f9f0 con 0x7ff548108610 2024-04-25T13:04:10.330 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.434+0000 7ff544ff9700 1 -- 172.21.15.72:0/2853267606 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7ff53802c3f0 con 0x7ff548108610 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.435+0000 7ff544ff9700 1 --2- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff5300394a0 0x7ff53003b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.435+0000 7ff544ff9700 1 -- 172.21.15.72:0/2853267606 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff53804edc0 con 0x7ff548108610 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.435+0000 7ff546ffd700 1 --2- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff5300394a0 0x7ff53003b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.435+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54810cd40 con 0x7ff548108610 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.435+0000 7ff546ffd700 1 --2- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff5300394a0 0x7ff53003b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006fd0 tx=0x7ff53c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.440+0000 7ff544ff9700 1 -- 172.21.15.72:0/2853267606 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff53801f030 con 0x7ff548108610 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.454+0000 7ff544ff9700 1 -- 172.21.15.72:0/2853267606 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7ff53802c780 con 0x7ff548108610 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.654+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff548104e70 con 0x7ff5300394a0 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.744+0000 7ff544ff9700 1 -- 172.21.15.72:0/2853267606 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff548104e70 con 0x7ff5300394a0 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff5300394a0 msgr2=0x7ff53003b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 --2- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff5300394a0 0x7ff53003b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006fd0 tx=0x7ff53c006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 msgr2=0x7ff54819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 --2- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff54819c530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5380119f0 tx=0x7ff538011ad0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.331 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 shutdown_connections 2024-04-25T13:04:10.332 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 --2- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff5300394a0 0x7ff53003b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.332 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 --2- 172.21.15.72:0/2853267606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff548108610 0x7ff54819c530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.332 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 >> 172.21.15.72:0/2853267606 conn(0x7ff548103c10 msgr2=0x7ff548104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:10.332 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 shutdown_connections 2024-04-25T13:04:10.332 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:09.746+0000 7ff54e134700 1 -- 172.21.15.72:0/2853267606 wait complete. 2024-04-25T13:04:10.332 INFO:teuthology.orchestra.run.smithi072.stdout:Creating initial admin user... 2024-04-25T13:04:10.632 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:10 smithi072 ceph-mon[29199]: mgrmap e12: x(active, since 2s) 2024-04-25T13:04:10.633 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:10 smithi072 ceph-mon[29199]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:10.633 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:10 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:10.633 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:10 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:10.633 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:10 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:12.329 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:12 smithi072 ceph-mon[29199]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:12.329 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:12 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:12.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$qZgLlNbtiJ1lrCev1PJaH.9oEF9067n5zSLcYyfY3Cc3ti5vn8Cqm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714050251, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.739+0000 7f3b1703c700 1 Processor -- start 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.739+0000 7f3b1703c700 1 -- start start 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.739+0000 7f3b1703c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b1010b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.739+0000 7f3b1703c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1007ce90 con 0x7f3b101090a0 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.739+0000 7f3b14dd8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b1010b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.739+0000 7f3b14dd8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b1010b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33390/0 (socket says 172.21.15.72:33390) 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.739+0000 7f3b14dd8700 1 -- 172.21.15.72:0/1176747135 learned_addr learned my addr 172.21.15.72:0/1176747135 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.740+0000 7f3b14dd8700 1 -- 172.21.15.72:0/1176747135 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1010b9d0 con 0x7f3b101090a0 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.740+0000 7f3b14dd8700 1 --2- 172.21.15.72:0/1176747135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b1010b490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b040098a0 tx=0x7f3b04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccfb8ce9fa0cbd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.740+0000 7f3b0f7fe700 1 -- 172.21.15.72:0/1176747135 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b04004160 con 0x7f3b101090a0 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.740+0000 7f3b0f7fe700 1 -- 172.21.15.72:0/1176747135 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b04004300 con 0x7f3b101090a0 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.740+0000 7f3b0f7fe700 1 -- 172.21.15.72:0/1176747135 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b04011a20 con 0x7f3b101090a0 2024-04-25T13:04:12.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.741+0000 7f3b1703c700 1 -- 172.21.15.72:0/1176747135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 msgr2=0x7f3b1010b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.741+0000 7f3b1703c700 1 --2- 172.21.15.72:0/1176747135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b1010b490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b040098a0 tx=0x7f3b04009bb0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.741+0000 7f3b1703c700 1 -- 172.21.15.72:0/1176747135 shutdown_connections 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.741+0000 7f3b1703c700 1 --2- 172.21.15.72:0/1176747135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b1010b490 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.741+0000 7f3b1703c700 1 -- 172.21.15.72:0/1176747135 >> 172.21.15.72:0/1176747135 conn(0x7f3b10102b10 msgr2=0x7f3b10104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.741+0000 7f3b1703c700 1 -- 172.21.15.72:0/1176747135 shutdown_connections 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.741+0000 7f3b1703c700 1 -- 172.21.15.72:0/1176747135 wait complete. 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b1703c700 1 Processor -- start 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b1703c700 1 -- start start 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b1703c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b10108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b1703c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1010c970 con 0x7f3b101090a0 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b14dd8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b10108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b14dd8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b10108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33406/0 (socket says 172.21.15.72:33406) 2024-04-25T13:04:12.338 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b14dd8700 1 -- 172.21.15.72:0/1085086446 learned_addr learned my addr 172.21.15.72:0/1085086446 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b14dd8700 1 -- 172.21.15.72:0/1085086446 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b10108e60 con 0x7f3b101090a0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b14dd8700 1 --2- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b10108920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b040116e0 tx=0x7f3b04011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.742+0000 7f3b0dffb700 1 -- 172.21.15.72:0/1085086446 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b04003b20 con 0x7f3b101090a0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.743+0000 7f3b0dffb700 1 -- 172.21.15.72:0/1085086446 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b0401b8f0 con 0x7f3b101090a0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.743+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b10107050 con 0x7f3b101090a0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.743+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b10107590 con 0x7f3b101090a0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.743+0000 7f3b0dffb700 1 -- 172.21.15.72:0/1085086446 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b040238c0 con 0x7f3b101090a0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.744+0000 7f3b0dffb700 1 -- 172.21.15.72:0/1085086446 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f3b04023b90 con 0x7f3b101090a0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.744+0000 7f3b0dffb700 1 --2- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3b000395b0 0x7f3b0003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.744+0000 7f3b0dffb700 1 -- 172.21.15.72:0/1085086446 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3b0404e280 con 0x7f3b101090a0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.744+0000 7f3b0ffff700 1 --2- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3b000395b0 0x7f3b0003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.339 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.744+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b100640a0 con 0x7f3b101090a0 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.744+0000 7f3b0ffff700 1 --2- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3b000395b0 0x7f3b0003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3afc006fd0 tx=0x7f3afc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.750+0000 7f3b0dffb700 1 -- 172.21.15.72:0/1085086446 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b0401f030 con 0x7f3b101090a0 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:10.975+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3b10104030 con 0x7f3b000395b0 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.197+0000 7f3b0dffb700 1 -- 172.21.15.72:0/1085086446 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3b10104030 con 0x7f3b000395b0 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.199+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3b000395b0 msgr2=0x7f3b0003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.199+0000 7f3b1703c700 1 --2- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3b000395b0 0x7f3b0003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3afc006fd0 tx=0x7f3afc006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.199+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 msgr2=0x7f3b10108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.199+0000 7f3b1703c700 1 --2- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b10108920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b040116e0 tx=0x7f3b04011aa0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.200+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 shutdown_connections 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.200+0000 7f3b1703c700 1 --2- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3b000395b0 0x7f3b0003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.200+0000 7f3b1703c700 1 --2- 172.21.15.72:0/1085086446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b101090a0 0x7f3b10108920 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.200+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 >> 172.21.15.72:0/1085086446 conn(0x7f3b10102b10 msgr2=0x7f3b10103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.200+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 shutdown_connections 2024-04-25T13:04:12.340 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:11.200+0000 7f3b1703c700 1 -- 172.21.15.72:0/1085086446 wait complete. 2024-04-25T13:04:12.341 INFO:teuthology.orchestra.run.smithi072.stdout:Fetching dashboard port number... 2024-04-25T13:04:13.502 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:13 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1362876301' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.737+0000 7f02b9072700 1 Processor -- start 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.737+0000 7f02b9072700 1 -- start start 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.737+0000 7f02b9072700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b4108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.737+0000 7f02b9072700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4109060 con 0x7f02b4108670 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.737+0000 7f02b2d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b4108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.737+0000 7f02b2d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b4108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33422/0 (socket says 172.21.15.72:33422) 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.737+0000 7f02b2d9d700 1 -- 172.21.15.72:0/917757394 learned_addr learned my addr 172.21.15.72:0/917757394 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.737+0000 7f02b2d9d700 1 -- 172.21.15.72:0/917757394 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b4109870 con 0x7f02b4108670 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.737+0000 7f02b2d9d700 1 --2- 172.21.15.72:0/917757394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b4108a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02a40098a0 tx=0x7f02a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5dedc16dd31afc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.510 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b1d9b700 1 -- 172.21.15.72:0/917757394 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02a4004160 con 0x7f02b4108670 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b1d9b700 1 -- 172.21.15.72:0/917757394 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02a4004300 con 0x7f02b4108670 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b1d9b700 1 -- 172.21.15.72:0/917757394 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02a4011a20 con 0x7f02b4108670 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b9072700 1 -- 172.21.15.72:0/917757394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 msgr2=0x7f02b4108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b9072700 1 --2- 172.21.15.72:0/917757394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b4108a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02a40098a0 tx=0x7f02a4009bb0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b9072700 1 -- 172.21.15.72:0/917757394 shutdown_connections 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b9072700 1 --2- 172.21.15.72:0/917757394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b4108a90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b9072700 1 -- 172.21.15.72:0/917757394 >> 172.21.15.72:0/917757394 conn(0x7f02b4103c30 msgr2=0x7f02b4106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b9072700 1 -- 172.21.15.72:0/917757394 shutdown_connections 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.738+0000 7f02b9072700 1 -- 172.21.15.72:0/917757394 wait complete. 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.739+0000 7f02b9072700 1 Processor -- start 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.739+0000 7f02b9072700 1 -- start start 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.739+0000 7f02b9072700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.511 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.739+0000 7f02b9072700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f02b410cc20 con 0x7f02b4108670 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.739+0000 7f02b2d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.739+0000 7f02b2d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33426/0 (socket says 172.21.15.72:33426) 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.739+0000 7f02b2d9d700 1 -- 172.21.15.72:0/1362876301 learned_addr learned my addr 172.21.15.72:0/1362876301 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.740+0000 7f02b2d9d700 1 -- 172.21.15.72:0/1362876301 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b419ca20 con 0x7f02b4108670 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.740+0000 7f02b2d9d700 1 --2- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b419c4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000c00 tx=0x7f02a4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.740+0000 7f029bfff700 1 -- 172.21.15.72:0/1362876301 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02a4003aa0 con 0x7f02b4108670 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.740+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b419ccb0 con 0x7f02b4108670 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.740+0000 7f029bfff700 1 -- 172.21.15.72:0/1362876301 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02a401b8f0 con 0x7f02b4108670 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.740+0000 7f029bfff700 1 -- 172.21.15.72:0/1362876301 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02a4023890 con 0x7f02b4108670 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.740+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b419f9a0 con 0x7f02b4108670 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.741+0000 7f029bfff700 1 -- 172.21.15.72:0/1362876301 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f02a402c3f0 con 0x7f02b4108670 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.741+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b40640a0 con 0x7f02b4108670 2024-04-25T13:04:13.512 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.741+0000 7f029bfff700 1 --2- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f029c039600 0x7f029c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.741+0000 7f029bfff700 1 -- 172.21.15.72:0/1362876301 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f02a404e130 con 0x7f02b4108670 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.741+0000 7f02b259c700 1 --2- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f029c039600 0x7f029c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.742+0000 7f02b259c700 1 --2- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f029c039600 0x7f029c03bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f02a8006fd0 tx=0x7f02a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.746+0000 7f029bfff700 1 -- 172.21.15.72:0/1362876301 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02a401f030 con 0x7f02b4108670 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.941+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f02b419fb70 con 0x7f02b4108670 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.941+0000 7f029bfff700 1 -- 172.21.15.72:0/1362876301 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f02a401b440 con 0x7f02b4108670 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.944+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f029c039600 msgr2=0x7f029c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.944+0000 7f02b9072700 1 --2- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f029c039600 0x7f029c03bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f02a8006fd0 tx=0x7f02a8006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.944+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 msgr2=0x7f02b419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.944+0000 7f02b9072700 1 --2- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b419c4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000c00 tx=0x7f02a4011aa0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.944+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 shutdown_connections 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.944+0000 7f02b9072700 1 --2- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f029c039600 0x7f029c03bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.945+0000 7f02b9072700 1 --2- 172.21.15.72:0/1362876301 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02b4108670 0x7f02b419c4e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.513 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.945+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 >> 172.21.15.72:0/1362876301 conn(0x7f02b4103c30 msgr2=0x7f02b4104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.514 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.945+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 shutdown_connections 2024-04-25T13:04:13.514 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:12.945+0000 7f02b9072700 1 -- 172.21.15.72:0/1362876301 wait complete. 2024-04-25T13:04:13.516 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:04:13.516 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-04-25T13:04:13.523 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:04:13.523 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-25T13:04:13.523 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-04-25T13:04:13.523 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T13:04:13.524 INFO:teuthology.orchestra.run.smithi072.stdout:Ceph Dashboard is now available at: 2024-04-25T13:04:13.524 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:13.524 INFO:teuthology.orchestra.run.smithi072.stdout: URL: https://smithi072:8443/ 2024-04-25T13:04:13.524 INFO:teuthology.orchestra.run.smithi072.stdout: User: admin 2024-04-25T13:04:13.525 INFO:teuthology.orchestra.run.smithi072.stdout: Password: suk27rc7ny 2024-04-25T13:04:13.525 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:13.525 INFO:teuthology.orchestra.run.smithi072.stdout:Saving cluster configuration to /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config directory 2024-04-25T13:04:13.525 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling autotune for osd_memory_target 2024-04-25T13:04:14.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:14 smithi072 ceph-mon[29199]: mgrmap e13: x(active, since 6s) 2024-04-25T13:04:14.732 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.960+0000 7fd3f2d4e700 1 Processor -- start 2024-04-25T13:04:14.732 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.960+0000 7fd3f2d4e700 1 -- start start 2024-04-25T13:04:14.732 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.961+0000 7fd3f2d4e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.961+0000 7fd3f2d4e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec109060 con 0x7fd3ec108670 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.961+0000 7fd3f0aea700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.961+0000 7fd3f0aea700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33438/0 (socket says 172.21.15.72:33438) 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.961+0000 7fd3f0aea700 1 -- 172.21.15.72:0/2332564601 learned_addr learned my addr 172.21.15.72:0/2332564601 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.961+0000 7fd3f0aea700 1 -- 172.21.15.72:0/2332564601 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec109820 con 0x7fd3ec108670 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.961+0000 7fd3f0aea700 1 --2- 172.21.15.72:0/2332564601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec108a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00098a0 tx=0x7fd3e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb236b4b383d3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.961+0000 7fd3eb7fe700 1 -- 172.21.15.72:0/2332564601 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e0004160 con 0x7fd3ec108670 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.962+0000 7fd3eb7fe700 1 -- 172.21.15.72:0/2332564601 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e0004300 con 0x7fd3ec108670 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.962+0000 7fd3eb7fe700 1 -- 172.21.15.72:0/2332564601 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e0011a20 con 0x7fd3ec108670 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.962+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2332564601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 msgr2=0x7fd3ec108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.962+0000 7fd3f2d4e700 1 --2- 172.21.15.72:0/2332564601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec108a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00098a0 tx=0x7fd3e0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.962+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2332564601 shutdown_connections 2024-04-25T13:04:14.733 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.962+0000 7fd3f2d4e700 1 --2- 172.21.15.72:0/2332564601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec108a90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.962+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2332564601 >> 172.21.15.72:0/2332564601 conn(0x7fd3ec103c30 msgr2=0x7fd3ec106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.962+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2332564601 shutdown_connections 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.962+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2332564601 wait complete. 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3f2d4e700 1 Processor -- start 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3f2d4e700 1 -- start start 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3f2d4e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3f2d4e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec10cbd0 con 0x7fd3ec108670 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3f0aea700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3f0aea700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33452/0 (socket says 172.21.15.72:33452) 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3f0aea700 1 -- 172.21.15.72:0/2841724109 learned_addr learned my addr 172.21.15.72:0/2841724109 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3f0aea700 1 -- 172.21.15.72:0/2841724109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec19c9f0 con 0x7fd3ec108670 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3f0aea700 1 --2- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec19c4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0000c00 tx=0x7fd3e0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.734 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.963+0000 7fd3e9ffb700 1 -- 172.21.15.72:0/2841724109 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e0003aa0 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.964+0000 7fd3e9ffb700 1 -- 172.21.15.72:0/2841724109 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e001b8f0 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.964+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ec19cc80 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.964+0000 7fd3e9ffb700 1 -- 172.21.15.72:0/2841724109 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e0023890 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.964+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ec19f970 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.965+0000 7fd3e9ffb700 1 -- 172.21.15.72:0/2841724109 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd3e001b440 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.965+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ec10cd50 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.965+0000 7fd3e9ffb700 1 --2- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3dc0396a0 0x7fd3dc03bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.965+0000 7fd3e9ffb700 1 -- 172.21.15.72:0/2841724109 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd3e004e070 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.965+0000 7fd3ebfff700 1 --2- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3dc0396a0 0x7fd3dc03bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.965+0000 7fd3ebfff700 1 --2- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3dc0396a0 0x7fd3dc03bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8006fd0 tx=0x7fd3d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:13.970+0000 7fd3e9ffb700 1 -- 172.21.15.72:0/2841724109 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3e001f030 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.166+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fd3ec19ffc0 con 0x7fd3ec108670 2024-04-25T13:04:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.167+0000 7fd3e9ffb700 1 -- 172.21.15.72:0/2841724109 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fd3e0016260 con 0x7fd3ec108670 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3dc0396a0 msgr2=0x7fd3dc03bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 --2- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3dc0396a0 0x7fd3dc03bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8006fd0 tx=0x7fd3d8006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 msgr2=0x7fd3ec19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 --2- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec19c4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0000c00 tx=0x7fd3e0011aa0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 shutdown_connections 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 --2- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3dc0396a0 0x7fd3dc03bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 --2- 172.21.15.72:0/2841724109 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3ec108670 0x7fd3ec19c4b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 >> 172.21.15.72:0/2841724109 conn(0x7fd3ec103c30 msgr2=0x7fd3ec104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 shutdown_connections 2024-04-25T13:04:14.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:14.169+0000 7fd3f2d4e700 1 -- 172.21.15.72:0/2841724109 wait complete. 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.148+0000 7f1c80d0a700 1 Processor -- start 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.148+0000 7f1c80d0a700 1 -- start start 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.148+0000 7f1c80d0a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c10b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.148+0000 7f1c80d0a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c07ce30 con 0x7f1c7c1090c0 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.148+0000 7f1c7a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c10b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.148+0000 7f1c7a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c10b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33460/0 (socket says 172.21.15.72:33460) 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.148+0000 7f1c7a59c700 1 -- 172.21.15.72:0/1567590098 learned_addr learned my addr 172.21.15.72:0/1567590098 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.148+0000 7f1c7a59c700 1 -- 172.21.15.72:0/1567590098 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c10b9f0 con 0x7f1c7c1090c0 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c7a59c700 1 --2- 172.21.15.72:0/1567590098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c10b4b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0098a0 tx=0x7f1c6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90e7d0b482f80c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:16.039 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c7959a700 1 -- 172.21.15.72:0/1567590098 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c6c004160 con 0x7f1c7c1090c0 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c7959a700 1 -- 172.21.15.72:0/1567590098 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c004300 con 0x7f1c7c1090c0 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c7959a700 1 -- 172.21.15.72:0/1567590098 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c6c011a20 con 0x7f1c7c1090c0 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c80d0a700 1 -- 172.21.15.72:0/1567590098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 msgr2=0x7f1c7c10b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c80d0a700 1 --2- 172.21.15.72:0/1567590098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c10b4b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0098a0 tx=0x7f1c6c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c80d0a700 1 -- 172.21.15.72:0/1567590098 shutdown_connections 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c80d0a700 1 --2- 172.21.15.72:0/1567590098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c10b4b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c80d0a700 1 -- 172.21.15.72:0/1567590098 >> 172.21.15.72:0/1567590098 conn(0x7f1c7c102a70 msgr2=0x7f1c7c104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c80d0a700 1 -- 172.21.15.72:0/1567590098 shutdown_connections 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.149+0000 7f1c80d0a700 1 -- 172.21.15.72:0/1567590098 wait complete. 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.150+0000 7f1c80d0a700 1 Processor -- start 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.150+0000 7f1c80d0a700 1 -- start start 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.150+0000 7f1c80d0a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.150+0000 7f1c80d0a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10c990 con 0x7f1c7c1090c0 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.150+0000 7f1c7a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.150+0000 7f1c7a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33468/0 (socket says 172.21.15.72:33468) 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.150+0000 7f1c7a59c700 1 -- 172.21.15.72:0/3249100864 learned_addr learned my addr 172.21.15.72:0/3249100864 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.151+0000 7f1c7a59c700 1 -- 172.21.15.72:0/3249100864 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c19c9e0 con 0x7f1c7c1090c0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.151+0000 7f1c7a59c700 1 --2- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c19c4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0116e0 tx=0x7f1c6c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.151+0000 7f1c6b7fe700 1 -- 172.21.15.72:0/3249100864 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c6c003b20 con 0x7f1c7c1090c0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.151+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c19cc70 con 0x7f1c7c1090c0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.151+0000 7f1c6b7fe700 1 -- 172.21.15.72:0/3249100864 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c01b8f0 con 0x7f1c7c1090c0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.151+0000 7f1c6b7fe700 1 -- 172.21.15.72:0/3249100864 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c6c0238c0 con 0x7f1c7c1090c0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.151+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c19f960 con 0x7f1c7c1090c0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.152+0000 7f1c6b7fe700 1 -- 172.21.15.72:0/3249100864 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1c6c023b70 con 0x7f1c7c1090c0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.152+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7c0640a0 con 0x7f1c7c1090c0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.152+0000 7f1c6b7fe700 1 --2- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1c64039650 0x7f1c6403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.152+0000 7f1c6b7fe700 1 -- 172.21.15.72:0/3249100864 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c6c04e510 con 0x7f1c7c1090c0 2024-04-25T13:04:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.152+0000 7f1c79d9b700 1 --2- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1c64039650 0x7f1c6403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.152+0000 7f1c79d9b700 1 --2- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1c64039650 0x7f1c6403bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c70006fd0 tx=0x7f1c70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.157+0000 7f1c6b7fe700 1 -- 172.21.15.72:0/3249100864 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c6c01f030 con 0x7f1c7c1090c0 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.463+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1c7c19fb30 con 0x7f1c7c1090c0 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.466+0000 7f1c6b7fe700 1 -- 172.21.15.72:0/3249100864 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f1c6c018c10 con 0x7f1c7c1090c0 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1c64039650 msgr2=0x7f1c6403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 --2- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1c64039650 0x7f1c6403bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c70006fd0 tx=0x7f1c70006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 msgr2=0x7f1c7c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 --2- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c19c4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0116e0 tx=0x7f1c6c011aa0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 shutdown_connections 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 --2- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1c64039650 0x7f1c6403bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 --2- 172.21.15.72:0/3249100864 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c7c1090c0 0x7f1c7c19c4a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 >> 172.21.15.72:0/3249100864 conn(0x7f1c7c102a70 msgr2=0x7f1c7c1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 shutdown_connections 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-25T13:04:15.468+0000 7f1c80d0a700 1 -- 172.21.15.72:0/3249100864 wait complete. 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: ceph telemetry on 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:For more information see: 2024-04-25T13:04:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T13:04:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:Bootstrap complete. 2024-04-25T13:04:16.075 INFO:tasks.cephadm:Fetching config... 2024-04-25T13:04:16.075 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:04:16.075 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T13:04:16.090 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T13:04:16.090 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:04:16.090 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T13:04:16.147 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T13:04:16.147 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:04:16.147 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T13:04:16.211 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T13:04:16.211 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:04:16.211 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T13:04:16.265 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T13:04:16.265 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUs9VyOnU4opZcEE4SjeT91WeK+Rv42I8BuXDuM+oRAd/wcIDdvhe5va+qs+XgU9KifRWF+VKRuYD6wKBXl+s/dTFkTY4zpQGr+x/02mHBdGCxcyWv0mryiuneX1G8wB1GruAJGV2waXkFjIWjTFU8fh8yhGzr/+fck6I5hB8mlugLA/0ODuz1nZeO1cOqYtOG+zi892orWwNdTejFoUSvTS/SZctonxdj+YjMwaOnW1AXixGT48u2C5XfFpxZuRVR8DOsGfRYX3Jk2vO7wkjg1BPZ0FXoULFXalEE+RfWrG9IrcMs+nMr+2vMSVhTyzuXzksf3LL/TG5CbW7wn7MDlG/TpAAthGojUMYYxzToIVCBV4XUYPrs9v4EQ0nBIpus61ythb/SKQD3vJBy0+5ynJwSYeaGeQxwQIDMpdhf4gSgB0mrmHZPRVNnpf2uH/gHYISUKI+IoS6H0+OjhvnvwCQPQtmdS/PAUu/8haU4p6CBQN90oywpnSHrdqJY0U0= ceph-d8ca43ea-0303-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:04:16.339 INFO:teuthology.orchestra.run.smithi072.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUs9VyOnU4opZcEE4SjeT91WeK+Rv42I8BuXDuM+oRAd/wcIDdvhe5va+qs+XgU9KifRWF+VKRuYD6wKBXl+s/dTFkTY4zpQGr+x/02mHBdGCxcyWv0mryiuneX1G8wB1GruAJGV2waXkFjIWjTFU8fh8yhGzr/+fck6I5hB8mlugLA/0ODuz1nZeO1cOqYtOG+zi892orWwNdTejFoUSvTS/SZctonxdj+YjMwaOnW1AXixGT48u2C5XfFpxZuRVR8DOsGfRYX3Jk2vO7wkjg1BPZ0FXoULFXalEE+RfWrG9IrcMs+nMr+2vMSVhTyzuXzksf3LL/TG5CbW7wn7MDlG/TpAAthGojUMYYxzToIVCBV4XUYPrs9v4EQ0nBIpus61ythb/SKQD3vJBy0+5ynJwSYeaGeQxwQIDMpdhf4gSgB0mrmHZPRVNnpf2uH/gHYISUKI+IoS6H0+OjhvnvwCQPQtmdS/PAUu/8haU4p6CBQN90oywpnSHrdqJY0U0= ceph-d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:04:16.349 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUs9VyOnU4opZcEE4SjeT91WeK+Rv42I8BuXDuM+oRAd/wcIDdvhe5va+qs+XgU9KifRWF+VKRuYD6wKBXl+s/dTFkTY4zpQGr+x/02mHBdGCxcyWv0mryiuneX1G8wB1GruAJGV2waXkFjIWjTFU8fh8yhGzr/+fck6I5hB8mlugLA/0ODuz1nZeO1cOqYtOG+zi892orWwNdTejFoUSvTS/SZctonxdj+YjMwaOnW1AXixGT48u2C5XfFpxZuRVR8DOsGfRYX3Jk2vO7wkjg1BPZ0FXoULFXalEE+RfWrG9IrcMs+nMr+2vMSVhTyzuXzksf3LL/TG5CbW7wn7MDlG/TpAAthGojUMYYxzToIVCBV4XUYPrs9v4EQ0nBIpus61ythb/SKQD3vJBy0+5ynJwSYeaGeQxwQIDMpdhf4gSgB0mrmHZPRVNnpf2uH/gHYISUKI+IoS6H0+OjhvnvwCQPQtmdS/PAUu/8haU4p6CBQN90oywpnSHrdqJY0U0= ceph-d8ca43ea-0303-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:04:16.387 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUs9VyOnU4opZcEE4SjeT91WeK+Rv42I8BuXDuM+oRAd/wcIDdvhe5va+qs+XgU9KifRWF+VKRuYD6wKBXl+s/dTFkTY4zpQGr+x/02mHBdGCxcyWv0mryiuneX1G8wB1GruAJGV2waXkFjIWjTFU8fh8yhGzr/+fck6I5hB8mlugLA/0ODuz1nZeO1cOqYtOG+zi892orWwNdTejFoUSvTS/SZctonxdj+YjMwaOnW1AXixGT48u2C5XfFpxZuRVR8DOsGfRYX3Jk2vO7wkjg1BPZ0FXoULFXalEE+RfWrG9IrcMs+nMr+2vMSVhTyzuXzksf3LL/TG5CbW7wn7MDlG/TpAAthGojUMYYxzToIVCBV4XUYPrs9v4EQ0nBIpus61ythb/SKQD3vJBy0+5ynJwSYeaGeQxwQIDMpdhf4gSgB0mrmHZPRVNnpf2uH/gHYISUKI+IoS6H0+OjhvnvwCQPQtmdS/PAUu/8haU4p6CBQN90oywpnSHrdqJY0U0= ceph-d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:04:16.399 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUs9VyOnU4opZcEE4SjeT91WeK+Rv42I8BuXDuM+oRAd/wcIDdvhe5va+qs+XgU9KifRWF+VKRuYD6wKBXl+s/dTFkTY4zpQGr+x/02mHBdGCxcyWv0mryiuneX1G8wB1GruAJGV2waXkFjIWjTFU8fh8yhGzr/+fck6I5hB8mlugLA/0ODuz1nZeO1cOqYtOG+zi892orWwNdTejFoUSvTS/SZctonxdj+YjMwaOnW1AXixGT48u2C5XfFpxZuRVR8DOsGfRYX3Jk2vO7wkjg1BPZ0FXoULFXalEE+RfWrG9IrcMs+nMr+2vMSVhTyzuXzksf3LL/TG5CbW7wn7MDlG/TpAAthGojUMYYxzToIVCBV4XUYPrs9v4EQ0nBIpus61ythb/SKQD3vJBy0+5ynJwSYeaGeQxwQIDMpdhf4gSgB0mrmHZPRVNnpf2uH/gHYISUKI+IoS6H0+OjhvnvwCQPQtmdS/PAUu/8haU4p6CBQN90oywpnSHrdqJY0U0= ceph-d8ca43ea-0303-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:04:16.439 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUs9VyOnU4opZcEE4SjeT91WeK+Rv42I8BuXDuM+oRAd/wcIDdvhe5va+qs+XgU9KifRWF+VKRuYD6wKBXl+s/dTFkTY4zpQGr+x/02mHBdGCxcyWv0mryiuneX1G8wB1GruAJGV2waXkFjIWjTFU8fh8yhGzr/+fck6I5hB8mlugLA/0ODuz1nZeO1cOqYtOG+zi892orWwNdTejFoUSvTS/SZctonxdj+YjMwaOnW1AXixGT48u2C5XfFpxZuRVR8DOsGfRYX3Jk2vO7wkjg1BPZ0FXoULFXalEE+RfWrG9IrcMs+nMr+2vMSVhTyzuXzksf3LL/TG5CbW7wn7MDlG/TpAAthGojUMYYxzToIVCBV4XUYPrs9v4EQ0nBIpus61ythb/SKQD3vJBy0+5ynJwSYeaGeQxwQIDMpdhf4gSgB0mrmHZPRVNnpf2uH/gHYISUKI+IoS6H0+OjhvnvwCQPQtmdS/PAUu/8haU4p6CBQN90oywpnSHrdqJY0U0= ceph-d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:04:16.451 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T13:04:16.688 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:16.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:16 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3249100864' entity='client.admin' 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.541+0000 7fdec1ddc700 1 -- 172.21.15.72:0/1552502709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 msgr2=0x7fdebc105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.541+0000 7fdec1ddc700 1 --2- 172.21.15.72:0/1552502709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 0x7fdebc105370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009910 tx=0x7fdeac009c20 comp rx=0 tx=0).stop 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.541+0000 7fdec1ddc700 1 -- 172.21.15.72:0/1552502709 shutdown_connections 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.541+0000 7fdec1ddc700 1 --2- 172.21.15.72:0/1552502709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 0x7fdebc105370 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.541+0000 7fdec1ddc700 1 -- 172.21.15.72:0/1552502709 >> 172.21.15.72:0/1552502709 conn(0x7fdebc0fc950 msgr2=0x7fdebc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.541+0000 7fdec1ddc700 1 -- 172.21.15.72:0/1552502709 shutdown_connections 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.541+0000 7fdec1ddc700 1 -- 172.21.15.72:0/1552502709 wait complete. 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.542+0000 7fdec1ddc700 1 Processor -- start 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.542+0000 7fdec1ddc700 1 -- start start 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.542+0000 7fdec1ddc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 0x7fdebc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:19.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.542+0000 7fdec1ddc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1083d0 con 0x7fdebc102f80 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.542+0000 7fdebb7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 0x7fdebc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.542+0000 7fdebb7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 0x7fdebc197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42466/0 (socket says 172.21.15.72:42466) 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.542+0000 7fdebb7fe700 1 -- 172.21.15.72:0/758855568 learned_addr learned my addr 172.21.15.72:0/758855568 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.542+0000 7fdebb7fe700 1 -- 172.21.15.72:0/758855568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc197f80 con 0x7fdebc102f80 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.543+0000 7fdebb7fe700 1 --2- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 0x7fdebc197a40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005190 tx=0x7fdeac004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.543+0000 7fdeb8ff9700 1 -- 172.21.15.72:0/758855568 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeac027070 con 0x7fdebc102f80 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.543+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc198270 con 0x7fdebc102f80 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.543+0000 7fdeb8ff9700 1 -- 172.21.15.72:0/758855568 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac022470 con 0x7fdebc102f80 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.543+0000 7fdeb8ff9700 1 -- 172.21.15.72:0/758855568 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeac00f5f0 con 0x7fdebc102f80 2024-04-25T13:04:19.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.543+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc19af00 con 0x7fdebc102f80 2024-04-25T13:04:19.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.543+0000 7fdeb8ff9700 1 -- 172.21.15.72:0/758855568 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdeac02a070 con 0x7fdebc102f80 2024-04-25T13:04:19.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.544+0000 7fdeb8ff9700 1 --2- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdea4039650 0x7fdea403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:19.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.544+0000 7fdeb8ff9700 1 -- 172.21.15.72:0/758855568 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdeac01e030 con 0x7fdebc102f80 2024-04-25T13:04:19.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.544+0000 7fdebaffd700 1 --2- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdea4039650 0x7fdea403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:19.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.544+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc108b50 con 0x7fdebc102f80 2024-04-25T13:04:19.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.544+0000 7fdebaffd700 1 --2- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdea4039650 0x7fdea403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0006fd0 tx=0x7fdeb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:19.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.549+0000 7fdeb8ff9700 1 -- 172.21.15.72:0/758855568 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdeac031030 con 0x7fdebc102f80 2024-04-25T13:04:19.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.749+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fdebc02e020 con 0x7fdebc102f80 2024-04-25T13:04:19.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.751+0000 7fdeb8ff9700 1 -- 172.21.15.72:0/758855568 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fdeac013070 con 0x7fdebc102f80 2024-04-25T13:04:19.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdea4039650 msgr2=0x7fdea403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:19.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 --2- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdea4039650 0x7fdea403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0006fd0 tx=0x7fdeb0006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:19.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 msgr2=0x7fdebc197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:19.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 --2- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 0x7fdebc197a40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005190 tx=0x7fdeac004f80 comp rx=0 tx=0).stop 2024-04-25T13:04:19.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 shutdown_connections 2024-04-25T13:04:19.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 --2- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdea4039650 0x7fdea403bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 --2- 172.21.15.72:0/758855568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdebc102f80 0x7fdebc197a40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 >> 172.21.15.72:0/758855568 conn(0x7fdebc0fc950 msgr2=0x7fdebc0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:19.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 shutdown_connections 2024-04-25T13:04:19.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:19.754+0000 7fdec1ddc700 1 -- 172.21.15.72:0/758855568 wait complete. 2024-04-25T13:04:20.443 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T13:04:20.443 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T13:04:20.672 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:21.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:20 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/758855568' entity='client.admin' 2024-04-25T13:04:21.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:21.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:21.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi072", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:21.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:21.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:21.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:22.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.185+0000 7f731b8f1700 1 -- 172.21.15.72:0/1549719700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 msgr2=0x7f73141017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:22.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.185+0000 7f731b8f1700 1 --2- 172.21.15.72:0/1549719700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 0x7f73141017c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7310009910 tx=0x7f7310009c20 comp rx=0 tx=0).stop 2024-04-25T13:04:22.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.185+0000 7f731b8f1700 1 -- 172.21.15.72:0/1549719700 shutdown_connections 2024-04-25T13:04:22.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.185+0000 7f731b8f1700 1 --2- 172.21.15.72:0/1549719700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 0x7f73141017c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.185+0000 7f731b8f1700 1 -- 172.21.15.72:0/1549719700 >> 172.21.15.72:0/1549719700 conn(0x7f73140fc940 msgr2=0x7f73140fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:22.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.185+0000 7f731b8f1700 1 -- 172.21.15.72:0/1549719700 shutdown_connections 2024-04-25T13:04:22.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.185+0000 7f731b8f1700 1 -- 172.21.15.72:0/1549719700 wait complete. 2024-04-25T13:04:22.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731b8f1700 1 Processor -- start 2024-04-25T13:04:22.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731b8f1700 1 -- start start 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731b8f1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 0x7f731406a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731b8f1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f73141072e0 con 0x7f73141013a0 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731968d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 0x7f731406a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731968d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 0x7f731406a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42490/0 (socket says 172.21.15.72:42490) 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731968d700 1 -- 172.21.15.72:0/3595826113 learned_addr learned my addr 172.21.15.72:0/3595826113 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731968d700 1 -- 172.21.15.72:0/3595826113 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731419a840 con 0x7f73141013a0 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731968d700 1 --2- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 0x7f731406a8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7310005800 tx=0x7f7310009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f730a7fc700 1 -- 172.21.15.72:0/3595826113 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7310027070 con 0x7f73141013a0 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f730a7fc700 1 -- 172.21.15.72:0/3595826113 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731000be10 con 0x7f73141013a0 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731419aa10 con 0x7f73141013a0 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.186+0000 7f730a7fc700 1 -- 172.21.15.72:0/3595826113 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f731000f460 con 0x7f73141013a0 2024-04-25T13:04:22.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.187+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731419adb0 con 0x7f73141013a0 2024-04-25T13:04:22.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.187+0000 7f730a7fc700 1 -- 172.21.15.72:0/3595826113 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f731002a070 con 0x7f73141013a0 2024-04-25T13:04:22.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.187+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7314107a60 con 0x7f73141013a0 2024-04-25T13:04:22.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.187+0000 7f730a7fc700 1 --2- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7300039650 0x7f730003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:22.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.188+0000 7f730a7fc700 1 -- 172.21.15.72:0/3595826113 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f731001e030 con 0x7f73141013a0 2024-04-25T13:04:22.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.188+0000 7f7318e8c700 1 --2- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7300039650 0x7f730003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:22.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.188+0000 7f7318e8c700 1 --2- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7300039650 0x7f730003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7304006fd0 tx=0x7f7304006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:22.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.193+0000 7f730a7fc700 1 -- 172.21.15.72:0/3595826113 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7310013070 con 0x7f73141013a0 2024-04-25T13:04:22.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.394+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f731402e240 con 0x7f7300039650 2024-04-25T13:04:22.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.398+0000 7f730a7fc700 1 -- 172.21.15.72:0/3595826113 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f731402e240 con 0x7f7300039650 2024-04-25T13:04:22.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7300039650 msgr2=0x7f730003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:22.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 --2- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7300039650 0x7f730003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7304006fd0 tx=0x7f7304006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:22.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 msgr2=0x7f731406a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:22.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 --2- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 0x7f731406a8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7310005800 tx=0x7f7310009f60 comp rx=0 tx=0).stop 2024-04-25T13:04:22.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 shutdown_connections 2024-04-25T13:04:22.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 --2- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7300039650 0x7f730003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 --2- 172.21.15.72:0/3595826113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73141013a0 0x7f731406a8e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 >> 172.21.15.72:0/3595826113 conn(0x7f73140fc940 msgr2=0x7f73140fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:22.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 shutdown_connections 2024-04-25T13:04:22.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:22.400+0000 7f731b8f1700 1 -- 172.21.15.72:0/3595826113 wait complete. 2024-04-25T13:04:23.182 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi164 2024-04-25T13:04:23.183 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T13:04:23.183 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:04:23.201 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T13:04:23.201 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:23.259 INFO:tasks.cephadm:Adding host smithi164 to orchestrator... 2024-04-25T13:04:23.259 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch host add smithi164 2024-04-25T13:04:23.499 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:23.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:23.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:24.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.971+0000 7f89abb41700 1 -- 172.21.15.72:0/1075739768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 msgr2=0x7f89a4101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:24.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.971+0000 7f89abb41700 1 --2- 172.21.15.72:0/1075739768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 0x7f89a4101990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009910 tx=0x7f89a0009c20 comp rx=0 tx=0).stop 2024-04-25T13:04:24.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.971+0000 7f89abb41700 1 -- 172.21.15.72:0/1075739768 shutdown_connections 2024-04-25T13:04:24.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.971+0000 7f89abb41700 1 --2- 172.21.15.72:0/1075739768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 0x7f89a4101990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:24.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.971+0000 7f89abb41700 1 -- 172.21.15.72:0/1075739768 >> 172.21.15.72:0/1075739768 conn(0x7f89a40fce40 msgr2=0x7f89a40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:24.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.971+0000 7f89abb41700 1 -- 172.21.15.72:0/1075739768 shutdown_connections 2024-04-25T13:04:24.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.971+0000 7f89abb41700 1 -- 172.21.15.72:0/1075739768 wait complete. 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.971+0000 7f89abb41700 1 Processor -- start 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89abb41700 1 -- start start 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89abb41700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 0x7f89a4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89abb41700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f89a410b830 con 0x7f89a41015b0 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89a98dd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 0x7f89a4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89a98dd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 0x7f89a4196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42500/0 (socket says 172.21.15.72:42500) 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89a98dd700 1 -- 172.21.15.72:0/471638282 learned_addr learned my addr 172.21.15.72:0/471638282 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89a98dd700 1 -- 172.21.15.72:0/471638282 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a41972d0 con 0x7f89a41015b0 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89a98dd700 1 --2- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 0x7f89a4196d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f89a00056d0 tx=0x7f89a00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f899affd700 1 -- 172.21.15.72:0/471638282 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89a0026070 con 0x7f89a41015b0 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a41975c0 con 0x7f89a41015b0 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f899affd700 1 -- 172.21.15.72:0/471638282 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89a000bba0 con 0x7f89a41015b0 2024-04-25T13:04:24.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f899affd700 1 -- 172.21.15.72:0/471638282 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89a0003920 con 0x7f89a41015b0 2024-04-25T13:04:24.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.972+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a419b660 con 0x7f89a41015b0 2024-04-25T13:04:24.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.973+0000 7f899affd700 1 -- 172.21.15.72:0/471638282 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f89a0029070 con 0x7f89a41015b0 2024-04-25T13:04:24.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.973+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a4102a90 con 0x7f89a41015b0 2024-04-25T13:04:24.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.973+0000 7f899affd700 1 --2- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8990039650 0x7f899003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:24.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.973+0000 7f899affd700 1 -- 172.21.15.72:0/471638282 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f89a001e070 con 0x7f89a41015b0 2024-04-25T13:04:24.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.974+0000 7f89a90dc700 1 --2- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8990039650 0x7f899003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:24.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.974+0000 7f89a90dc700 1 --2- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8990039650 0x7f899003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8994006fd0 tx=0x7f8994006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:24.977 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:24.979+0000 7f899affd700 1 -- 172.21.15.72:0/471638282 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89a0013070 con 0x7f89a41015b0 2024-04-25T13:04:25.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:25.182+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}) -- 0x7f89a41980f0 con 0x7f8990039650 2024-04-25T13:04:25.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:25 smithi072 ceph-mon[29199]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:26.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:26 smithi072 ceph-mon[29199]: Deploying cephadm binary to smithi164 2024-04-25T13:04:27.733 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:27 smithi072 ceph-mon[29199]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:29.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.318+0000 7f899affd700 1 -- 172.21.15.72:0/471638282 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f89a41980f0 con 0x7f8990039650 2024-04-25T13:04:29.317 INFO:teuthology.orchestra.run.smithi072.stdout:Added host 'smithi164' with addr '172.21.15.164' 2024-04-25T13:04:29.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.320+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8990039650 msgr2=0x7f899003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:29.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.320+0000 7f89abb41700 1 --2- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8990039650 0x7f899003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8994006fd0 tx=0x7f8994006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:29.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.320+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 msgr2=0x7f89a4196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:29.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.320+0000 7f89abb41700 1 --2- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 0x7f89a4196d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f89a00056d0 tx=0x7f89a00057b0 comp rx=0 tx=0).stop 2024-04-25T13:04:29.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.320+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 shutdown_connections 2024-04-25T13:04:29.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.320+0000 7f89abb41700 1 --2- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8990039650 0x7f899003bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:29.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.320+0000 7f89abb41700 1 --2- 172.21.15.72:0/471638282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89a41015b0 0x7f89a4196d90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:29.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.320+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 >> 172.21.15.72:0/471638282 conn(0x7f89a40fce40 msgr2=0x7f89a4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:29.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.321+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 shutdown_connections 2024-04-25T13:04:29.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:29.321+0000 7f89abb41700 1 -- 172.21.15.72:0/471638282 wait complete. 2024-04-25T13:04:29.900 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:04:30.131 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:30.483 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:30 smithi072 ceph-mon[29199]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:30.483 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:30.483 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:30 smithi072 ceph-mon[29199]: Added host smithi164 2024-04-25T13:04:30.483 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:30.483 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:31.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.586+0000 7f7e3e441700 1 -- 172.21.15.72:0/3778842388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 msgr2=0x7f7e381088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:31.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.586+0000 7f7e3e441700 1 --2- 172.21.15.72:0/3778842388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 0x7f7e381088c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009910 tx=0x7f7e28009c20 comp rx=0 tx=0).stop 2024-04-25T13:04:31.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.586+0000 7f7e3e441700 1 -- 172.21.15.72:0/3778842388 shutdown_connections 2024-04-25T13:04:31.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.586+0000 7f7e3e441700 1 --2- 172.21.15.72:0/3778842388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 0x7f7e381088c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.586+0000 7f7e3e441700 1 -- 172.21.15.72:0/3778842388 >> 172.21.15.72:0/3778842388 conn(0x7f7e380fdf80 msgr2=0x7f7e381003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:31.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.586+0000 7f7e3e441700 1 -- 172.21.15.72:0/3778842388 shutdown_connections 2024-04-25T13:04:31.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.586+0000 7f7e3e441700 1 -- 172.21.15.72:0/3778842388 wait complete. 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.587+0000 7f7e3e441700 1 Processor -- start 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.587+0000 7f7e3e441700 1 -- start start 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.587+0000 7f7e3e441700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 0x7f7e38197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.587+0000 7f7e3e441700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810c940 con 0x7f7e381084e0 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.587+0000 7f7e37fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 0x7f7e38197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.587+0000 7f7e37fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 0x7f7e38197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57854/0 (socket says 172.21.15.72:57854) 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.587+0000 7f7e37fff700 1 -- 172.21.15.72:0/1808552321 learned_addr learned my addr 172.21.15.72:0/1808552321 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.588+0000 7f7e37fff700 1 -- 172.21.15.72:0/1808552321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e381983f0 con 0x7f7e381084e0 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.588+0000 7f7e37fff700 1 --2- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 0x7f7e38197eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e280056d0 tx=0x7f7e280057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.588+0000 7f7e357fa700 1 -- 172.21.15.72:0/1808552321 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e28026070 con 0x7f7e381084e0 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.588+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e381986e0 con 0x7f7e381084e0 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.588+0000 7f7e357fa700 1 -- 172.21.15.72:0/1808552321 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e2800bba0 con 0x7f7e381084e0 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.588+0000 7f7e357fa700 1 -- 172.21.15.72:0/1808552321 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e28003920 con 0x7f7e381084e0 2024-04-25T13:04:31.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.588+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3819c780 con 0x7f7e381084e0 2024-04-25T13:04:31.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.589+0000 7f7e357fa700 1 -- 172.21.15.72:0/1808552321 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7e28029070 con 0x7f7e381084e0 2024-04-25T13:04:31.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.589+0000 7f7e357fa700 1 --2- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7e20039650 0x7f7e2003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:31.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.589+0000 7f7e357fa700 1 -- 172.21.15.72:0/1808552321 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e2801e070 con 0x7f7e381084e0 2024-04-25T13:04:31.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.589+0000 7f7e377fe700 1 --2- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7e20039650 0x7f7e2003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:31.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.589+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e38103b90 con 0x7f7e381084e0 2024-04-25T13:04:31.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.590+0000 7f7e377fe700 1 --2- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7e20039650 0x7f7e2003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c006fd0 tx=0x7f7e2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:31.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.594+0000 7f7e357fa700 1 -- 172.21.15.72:0/1808552321 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e28030070 con 0x7f7e381084e0 2024-04-25T13:04:31.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:31.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:31 smithi072 ceph-mon[29199]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:31.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.796+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e38199210 con 0x7f7e20039650 2024-04-25T13:04:31.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.798+0000 7f7e357fa700 1 -- 172.21.15.72:0/1808552321 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f7e38199210 con 0x7f7e20039650 2024-04-25T13:04:31.797 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:31.797 INFO:teuthology.orchestra.run.smithi072.stdout:[{"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}] 2024-04-25T13:04:31.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7e20039650 msgr2=0x7f7e2003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:31.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 --2- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7e20039650 0x7f7e2003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c006fd0 tx=0x7f7e2c006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:31.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 msgr2=0x7f7e38197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:31.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 --2- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 0x7f7e38197eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e280056d0 tx=0x7f7e280057b0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 shutdown_connections 2024-04-25T13:04:31.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 --2- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7e20039650 0x7f7e2003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 --2- 172.21.15.72:0/1808552321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e381084e0 0x7f7e38197eb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 >> 172.21.15.72:0/1808552321 conn(0x7f7e380fdf80 msgr2=0x7f7e3810a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:31.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 shutdown_connections 2024-04-25T13:04:31.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:31.800+0000 7f7e3e441700 1 -- 172.21.15.72:0/1808552321 wait complete. 2024-04-25T13:04:32.378 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-04-25T13:04:32.379 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T13:04:32.379 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:04:32.397 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T13:04:32.397 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:32.455 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-04-25T13:04:32.455 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch host add smithi193 2024-04-25T13:04:32.687 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:33.115 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:32 smithi072 ceph-mon[29199]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:04:33.115 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:33.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:33 smithi072 ceph-mon[29199]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:34.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.165+0000 7f23bb04c700 1 -- 172.21.15.72:0/2378856027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 msgr2=0x7f23b41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.165+0000 7f23bb04c700 1 --2- 172.21.15.72:0/2378856027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 0x7f23b41088c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23b000b0a0 tx=0x7f23b000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.165+0000 7f23bb04c700 1 -- 172.21.15.72:0/2378856027 shutdown_connections 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.165+0000 7f23bb04c700 1 --2- 172.21.15.72:0/2378856027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 0x7f23b41088c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.165+0000 7f23bb04c700 1 -- 172.21.15.72:0/2378856027 >> 172.21.15.72:0/2378856027 conn(0x7f23b4076ea0 msgr2=0x7f23b40772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.165+0000 7f23bb04c700 1 -- 172.21.15.72:0/2378856027 shutdown_connections 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.165+0000 7f23bb04c700 1 -- 172.21.15.72:0/2378856027 wait complete. 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.166+0000 7f23bb04c700 1 Processor -- start 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.166+0000 7f23bb04c700 1 -- start start 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.166+0000 7f23bb04c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 0x7f23b4101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.166+0000 7f23bb04c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f23b410c920 con 0x7f23b41084e0 2024-04-25T13:04:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.166+0000 7f23b8de8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 0x7f23b4101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.166+0000 7f23b8de8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 0x7f23b4101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57882/0 (socket says 172.21.15.72:57882) 2024-04-25T13:04:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.166+0000 7f23b8de8700 1 -- 172.21.15.72:0/484732205 learned_addr learned my addr 172.21.15.72:0/484732205 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.166+0000 7f23b8de8700 1 -- 172.21.15.72:0/484732205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b4103300 con 0x7f23b41084e0 2024-04-25T13:04:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.166+0000 7f23b8de8700 1 --2- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 0x7f23b4101460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f23b0006130 tx=0x7f23b00097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.167+0000 7f23a9ffb700 1 -- 172.21.15.72:0/484732205 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23b0021020 con 0x7f23b41084e0 2024-04-25T13:04:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.167+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b41019a0 con 0x7f23b41084e0 2024-04-25T13:04:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.167+0000 7f23a9ffb700 1 -- 172.21.15.72:0/484732205 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23b00079b0 con 0x7f23b41084e0 2024-04-25T13:04:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.167+0000 7f23a9ffb700 1 -- 172.21.15.72:0/484732205 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23b0017040 con 0x7f23b41084e0 2024-04-25T13:04:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.167+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b4101e60 con 0x7f23b41084e0 2024-04-25T13:04:34.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.168+0000 7f23a9ffb700 1 -- 172.21.15.72:0/484732205 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f23b0024020 con 0x7f23b41084e0 2024-04-25T13:04:34.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.168+0000 7f23a9ffb700 1 --2- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f239c039650 0x7f239c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:34.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.168+0000 7f23a9ffb700 1 -- 172.21.15.72:0/484732205 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f23b000f050 con 0x7f23b41084e0 2024-04-25T13:04:34.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.168+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b4103b90 con 0x7f23b41084e0 2024-04-25T13:04:34.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.168+0000 7f23abfff700 1 --2- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f239c039650 0x7f239c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:34.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.168+0000 7f23abfff700 1 --2- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f239c039650 0x7f239c03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f23a4006fd0 tx=0x7f23a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:34.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.173+0000 7f23a9ffb700 1 -- 172.21.15.72:0/484732205 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23b002b020 con 0x7f23b41084e0 2024-04-25T13:04:34.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:34.375+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) -- 0x7f23b4102940 con 0x7f239c039650 2024-04-25T13:04:36.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:36.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:36.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:36.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:36.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:36.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:36.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:36.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:36.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T13:04:36.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:36.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: Deploying cephadm binary to smithi193 2024-04-25T13:04:36.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:35 smithi072 ceph-mon[29199]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:37.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:36 smithi072 ceph-mon[29199]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:37.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:36 smithi072 ceph-mon[29199]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:38.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:37 smithi072 ceph-mon[29199]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:38.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.583+0000 7f23a9ffb700 1 -- 172.21.15.72:0/484732205 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f23b4102940 con 0x7f239c039650 2024-04-25T13:04:38.583 INFO:teuthology.orchestra.run.smithi072.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-04-25T13:04:38.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f239c039650 msgr2=0x7f239c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 --2- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f239c039650 0x7f239c03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f23a4006fd0 tx=0x7f23a4006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:38.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 msgr2=0x7f23b4101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 --2- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 0x7f23b4101460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f23b0006130 tx=0x7f23b00097d0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 shutdown_connections 2024-04-25T13:04:38.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 --2- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f239c039650 0x7f239c03bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 --2- 172.21.15.72:0/484732205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23b41084e0 0x7f23b4101460 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 >> 172.21.15.72:0/484732205 conn(0x7f23b4076ea0 msgr2=0x7f23b40fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:38.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 shutdown_connections 2024-04-25T13:04:38.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:38.586+0000 7f23bb04c700 1 -- 172.21.15.72:0/484732205 wait complete. 2024-04-25T13:04:39.170 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:39 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:39.170 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:39 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:39.170 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:39 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:39.170 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:39 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:39.170 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:39 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:39.170 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:39 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:39.197 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:04:39.428 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:40.425 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:40 smithi072 ceph-mon[29199]: Added host smithi193 2024-04-25T13:04:40.426 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:40 smithi072 ceph-mon[29199]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:40.426 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:40 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:40.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.906+0000 7f7d5887d700 1 -- 172.21.15.72:0/1806787740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 msgr2=0x7f7d541019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:40.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.906+0000 7f7d5887d700 1 --2- 172.21.15.72:0/1806787740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 0x7f7d541019d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c009910 tx=0x7f7d3c009c20 comp rx=0 tx=0).stop 2024-04-25T13:04:40.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.906+0000 7f7d5887d700 1 -- 172.21.15.72:0/1806787740 shutdown_connections 2024-04-25T13:04:40.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.906+0000 7f7d5887d700 1 --2- 172.21.15.72:0/1806787740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 0x7f7d541019d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:40.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.906+0000 7f7d5887d700 1 -- 172.21.15.72:0/1806787740 >> 172.21.15.72:0/1806787740 conn(0x7f7d54076ea0 msgr2=0x7f7d540772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:40.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.906+0000 7f7d5887d700 1 -- 172.21.15.72:0/1806787740 shutdown_connections 2024-04-25T13:04:40.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.906+0000 7f7d5887d700 1 -- 172.21.15.72:0/1806787740 wait complete. 2024-04-25T13:04:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.907+0000 7f7d5887d700 1 Processor -- start 2024-04-25T13:04:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.907+0000 7f7d5887d700 1 -- start start 2024-04-25T13:04:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.907+0000 7f7d5887d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 0x7f7d54198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.907+0000 7f7d5887d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410b820 con 0x7f7d541015f0 2024-04-25T13:04:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.907+0000 7f7d5259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 0x7f7d54198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.907+0000 7f7d5259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 0x7f7d54198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43470/0 (socket says 172.21.15.72:43470) 2024-04-25T13:04:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.907+0000 7f7d5259c700 1 -- 172.21.15.72:0/282391241 learned_addr learned my addr 172.21.15.72:0/282391241 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.907+0000 7f7d5259c700 1 -- 172.21.15.72:0/282391241 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d54199290 con 0x7f7d541015f0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.908+0000 7f7d5259c700 1 --2- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 0x7f7d54198d50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c006010 tx=0x7f7d3c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.908+0000 7f7d4b7fe700 1 -- 172.21.15.72:0/282391241 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d3c026070 con 0x7f7d541015f0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.908+0000 7f7d4b7fe700 1 -- 172.21.15.72:0/282391241 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d3c00bba0 con 0x7f7d541015f0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.908+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d54199580 con 0x7f7d541015f0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.908+0000 7f7d4b7fe700 1 -- 172.21.15.72:0/282391241 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d3c003910 con 0x7f7d541015f0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.908+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d54199a40 con 0x7f7d541015f0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.909+0000 7f7d4b7fe700 1 -- 172.21.15.72:0/282391241 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7d3c029070 con 0x7f7d541015f0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.909+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d54102a80 con 0x7f7d541015f0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.909+0000 7f7d4b7fe700 1 --2- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7d40039650 0x7f7d4003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.909+0000 7f7d4b7fe700 1 -- 172.21.15.72:0/282391241 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7d3c01e070 con 0x7f7d541015f0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.909+0000 7f7d51d9b700 1 --2- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7d40039650 0x7f7d4003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:40.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.909+0000 7f7d51d9b700 1 --2- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7d40039650 0x7f7d4003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d44006fd0 tx=0x7f7d44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:40.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:40.914+0000 7f7d4b7fe700 1 -- 172.21.15.72:0/282391241 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d3c013070 con 0x7f7d541015f0 2024-04-25T13:04:41.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.118+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d5406a420 con 0x7f7d40039650 2024-04-25T13:04:41.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.119+0000 7f7d4b7fe700 1 -- 172.21.15.72:0/282391241 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f7d5406a420 con 0x7f7d40039650 2024-04-25T13:04:41.118 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:04:41.118 INFO:teuthology.orchestra.run.smithi072.stdout:[{"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-04-25T13:04:41.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7d40039650 msgr2=0x7f7d4003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 --2- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7d40039650 0x7f7d4003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d44006fd0 tx=0x7f7d44006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:41.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 msgr2=0x7f7d54198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 --2- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 0x7f7d54198d50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c006010 tx=0x7f7d3c005780 comp rx=0 tx=0).stop 2024-04-25T13:04:41.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 shutdown_connections 2024-04-25T13:04:41.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 --2- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7d40039650 0x7f7d4003bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 --2- 172.21.15.72:0/282391241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d541015f0 0x7f7d54198d50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 >> 172.21.15.72:0/282391241 conn(0x7f7d54076ea0 msgr2=0x7f7d54102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:41.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 shutdown_connections 2024-04-25T13:04:41.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:41.121+0000 7f7d5887d700 1 -- 172.21.15.72:0/282391241 wait complete. 2024-04-25T13:04:41.745 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T13:04:41.745 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T13:04:41.978 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:42.388 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:42 smithi072 ceph-mon[29199]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:42.388 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:42 smithi072 ceph-mon[29199]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:04:42.388 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:42 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:43.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.479+0000 7f80084f2700 1 -- 172.21.15.72:0/3608262152 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 msgr2=0x7f8000107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:43.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.479+0000 7f80084f2700 1 --2- 172.21.15.72:0/3608262152 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 0x7f8000107820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009910 tx=0x7f7ff0009c20 comp rx=0 tx=0).stop 2024-04-25T13:04:43.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.479+0000 7f80084f2700 1 -- 172.21.15.72:0/3608262152 shutdown_connections 2024-04-25T13:04:43.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.479+0000 7f80084f2700 1 --2- 172.21.15.72:0/3608262152 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 0x7f8000107820 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:43.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.479+0000 7f80084f2700 1 -- 172.21.15.72:0/3608262152 >> 172.21.15.72:0/3608262152 conn(0x7f8000077af0 msgr2=0x7f8000077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:43.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.479+0000 7f80084f2700 1 -- 172.21.15.72:0/3608262152 shutdown_connections 2024-04-25T13:04:43.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.479+0000 7f80084f2700 1 -- 172.21.15.72:0/3608262152 wait complete. 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.480+0000 7f80084f2700 1 Processor -- start 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.480+0000 7f80084f2700 1 -- start start 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.480+0000 7f80084f2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 0x7f8000101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.480+0000 7f80084f2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f800010c960 con 0x7f8000107440 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.480+0000 7f800628e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 0x7f8000101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.480+0000 7f800628e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 0x7f8000101be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43498/0 (socket says 172.21.15.72:43498) 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.480+0000 7f800628e700 1 -- 172.21.15.72:0/3466995133 learned_addr learned my addr 172.21.15.72:0/3466995133 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.481+0000 7f800628e700 1 -- 172.21.15.72:0/3466995133 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8000103a80 con 0x7f8000107440 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.481+0000 7f800628e700 1 --2- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 0x7f8000101be0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00056d0 tx=0x7f7ff00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.481+0000 7f7ff77fe700 1 -- 172.21.15.72:0/3466995133 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff0026070 con 0x7f8000107440 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.481+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8000102120 con 0x7f8000107440 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.481+0000 7f7ff77fe700 1 -- 172.21.15.72:0/3466995133 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff000bc00 con 0x7f8000107440 2024-04-25T13:04:43.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.481+0000 7f7ff77fe700 1 -- 172.21.15.72:0/3466995133 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff0003920 con 0x7f8000107440 2024-04-25T13:04:43.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.481+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80001025e0 con 0x7f8000107440 2024-04-25T13:04:43.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.482+0000 7f7ff77fe700 1 -- 172.21.15.72:0/3466995133 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7ff0029070 con 0x7f8000107440 2024-04-25T13:04:43.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.482+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80000640a0 con 0x7f8000107440 2024-04-25T13:04:43.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.482+0000 7f7ff77fe700 1 --2- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7fec039650 0x7f7fec03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:43.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.482+0000 7f7ff77fe700 1 -- 172.21.15.72:0/3466995133 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7ff001e070 con 0x7f8000107440 2024-04-25T13:04:43.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.482+0000 7f8005a8d700 1 --2- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7fec039650 0x7f7fec03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:43.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.482+0000 7f8005a8d700 1 --2- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7fec039650 0x7f7fec03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc006fd0 tx=0x7f7ffc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:43.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.487+0000 7f7ff77fe700 1 -- 172.21.15.72:0/3466995133 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ff0013070 con 0x7f8000107440 2024-04-25T13:04:43.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:43.692+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8000103d50 con 0x7f8000107440 2024-04-25T13:04:44.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.178+0000 7f7ff77fe700 1 -- 172.21.15.72:0/3466995133 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8000103d50 con 0x7f8000107440 2024-04-25T13:04:44.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7fec039650 msgr2=0x7f7fec03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:44.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 --2- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7fec039650 0x7f7fec03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc006fd0 tx=0x7f7ffc006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:44.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 msgr2=0x7f8000101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:44.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 --2- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 0x7f8000101be0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00056d0 tx=0x7f7ff00057b0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 shutdown_connections 2024-04-25T13:04:44.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 --2- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7fec039650 0x7f7fec03bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 --2- 172.21.15.72:0/3466995133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8000107440 0x7f8000101be0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 >> 172.21.15.72:0/3466995133 conn(0x7f8000077af0 msgr2=0x7f8000104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:44.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 shutdown_connections 2024-04-25T13:04:44.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:04:44.181+0000 7f80084f2700 1 -- 172.21.15.72:0/3466995133 wait complete. 2024-04-25T13:04:44.180 INFO:teuthology.orchestra.run.smithi072.stderr:adjusted tunables profile to default 2024-04-25T13:04:44.508 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:44 smithi072 ceph-mon[29199]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:44.508 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:44 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3466995133' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T13:04:44.836 INFO:tasks.cephadm:Adding mon.a on smithi072 2024-04-25T13:04:44.836 INFO:tasks.cephadm:Adding mon.b on smithi164 2024-04-25T13:04:44.836 INFO:tasks.cephadm:Adding mon.c on smithi193 2024-04-25T13:04:44.836 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi072:172.21.15.72=a;smithi164:172.21.15.164=b;smithi193:172.21.15.193=c' 2024-04-25T13:04:45.072 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:45.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3466995133' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:46.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:46 smithi072 ceph-mon[29199]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:46.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:46 smithi072 ceph-mon[29199]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:46.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:46 smithi072 ceph-mon[29199]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:46.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:46 smithi072 ceph-mon[29199]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:46.608 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.608+0000 7f4023fb9700 1 -- 172.21.15.193:0/908266244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 msgr2=0x7f401c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:46.608 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.608+0000 7f4023fb9700 1 --2- 172.21.15.193:0/908266244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 0x7f401c1019c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4018009910 tx=0x7f4018009c20 comp rx=0 tx=0).stop 2024-04-25T13:04:46.608 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.608+0000 7f4023fb9700 1 -- 172.21.15.193:0/908266244 shutdown_connections 2024-04-25T13:04:46.608 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.608+0000 7f4023fb9700 1 --2- 172.21.15.193:0/908266244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 0x7f401c1019c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:46.608 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.608+0000 7f4023fb9700 1 -- 172.21.15.193:0/908266244 >> 172.21.15.193:0/908266244 conn(0x7f401c0fced0 msgr2=0x7f401c0ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:46.608 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.608+0000 7f4023fb9700 1 -- 172.21.15.193:0/908266244 shutdown_connections 2024-04-25T13:04:46.608 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.608+0000 7f4023fb9700 1 -- 172.21.15.193:0/908266244 wait complete. 2024-04-25T13:04:46.608 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.609+0000 7f4023fb9700 1 Processor -- start 2024-04-25T13:04:46.609 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.609+0000 7f4023fb9700 1 -- start start 2024-04-25T13:04:46.609 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.609+0000 7f4023fb9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 0x7f401c196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:46.609 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.609+0000 7f4023fb9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f401c10b860 con 0x7f401c1015e0 2024-04-25T13:04:46.609 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.609+0000 7f4021d55700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 0x7f401c196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:46.609 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.609+0000 7f4021d55700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 0x7f401c196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.193:39778/0 (socket says 172.21.15.193:39778) 2024-04-25T13:04:46.609 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.609+0000 7f4021d55700 1 -- 172.21.15.193:0/3441155364 learned_addr learned my addr 172.21.15.193:0/3441155364 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:04:46.610 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.610+0000 7f4021d55700 1 -- 172.21.15.193:0/3441155364 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c197310 con 0x7f401c1015e0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.610+0000 7f4021d55700 1 --2- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 0x7f401c196dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f40180056d0 tx=0x7f40180057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.610+0000 7f4012ffd700 1 -- 172.21.15.193:0/3441155364 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4018026070 con 0x7f401c1015e0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.610+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401c197600 con 0x7f401c1015e0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.610+0000 7f4012ffd700 1 -- 172.21.15.193:0/3441155364 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f401800bba0 con 0x7f401c1015e0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.610+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401c19b6a0 con 0x7f401c1015e0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.610+0000 7f4012ffd700 1 -- 172.21.15.193:0/3441155364 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4018003920 con 0x7f401c1015e0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.611+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401c102ac0 con 0x7f401c1015e0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.611+0000 7f4012ffd700 1 -- 172.21.15.193:0/3441155364 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4018029070 con 0x7f401c1015e0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.611+0000 7f4012ffd700 1 --2- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4008039650 0x7f400803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.611+0000 7f4012ffd700 1 -- 172.21.15.193:0/3441155364 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f401801e070 con 0x7f401c1015e0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.611+0000 7f4021554700 1 --2- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4008039650 0x7f400803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:46.611 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.612+0000 7f4021554700 1 --2- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4008039650 0x7f400803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f400c006fd0 tx=0x7f400c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:46.615 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.617+0000 7f4012ffd700 1 -- 172.21.15.193:0/3441155364 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4018013070 con 0x7f401c1015e0 2024-04-25T13:04:46.826 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.827+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi164:172.21.15.164=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}) -- 0x7f401c198130 con 0x7f4008039650 2024-04-25T13:04:46.833 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.834+0000 7f4012ffd700 1 -- 172.21.15.193:0/3441155364 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f401c198130 con 0x7f4008039650 2024-04-25T13:04:46.833 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-04-25T13:04:46.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4008039650 msgr2=0x7f400803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:46.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 --2- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4008039650 0x7f400803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f400c006fd0 tx=0x7f400c006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:46.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 msgr2=0x7f401c196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:46.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 --2- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 0x7f401c196dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f40180056d0 tx=0x7f40180057b0 comp rx=0 tx=0).stop 2024-04-25T13:04:46.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 shutdown_connections 2024-04-25T13:04:46.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 --2- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4008039650 0x7f400803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:46.836 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 --2- 172.21.15.193:0/3441155364 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f401c1015e0 0x7f401c196dd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:46.836 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 >> 172.21.15.193:0/3441155364 conn(0x7f401c0fced0 msgr2=0x7f401c102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:46.836 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 shutdown_connections 2024-04-25T13:04:46.836 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:46.836+0000 7f4023fb9700 1 -- 172.21.15.193:0/3441155364 wait complete. 2024-04-25T13:04:47.483 DEBUG:teuthology.orchestra.run.smithi164:mon.b> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T13:04:47.486 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T13:04:47.488 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:04:47.489 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:04:48.259 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:48.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi164:172.21.15.164=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: Saving service mon spec with placement smithi072:172.21.15.72=a;smithi164:172.21.15.164=b;smithi193:172.21.15.193=c;count:3 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: Deploying daemon mon.c on smithi193 2024-04-25T13:04:48.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:47 smithi072 ceph-mon[29199]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:50.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:50 smithi072 ceph-mon[29199]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:51.088 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 systemd[1]: Starting Ceph mon.c for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:04:51.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.412+0000 7f00e140e700 1 -- 172.21.15.193:0/993241063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 msgr2=0x7f00dc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:51.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.412+0000 7f00e140e700 1 --2- 172.21.15.193:0/993241063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 0x7f00dc101990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009910 tx=0x7f00cc009c20 comp rx=0 tx=0).stop 2024-04-25T13:04:51.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.412+0000 7f00e140e700 1 -- 172.21.15.193:0/993241063 shutdown_connections 2024-04-25T13:04:51.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.412+0000 7f00e140e700 1 --2- 172.21.15.193:0/993241063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 0x7f00dc101990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:51.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.412+0000 7f00e140e700 1 -- 172.21.15.193:0/993241063 >> 172.21.15.193:0/993241063 conn(0x7f00dc0fce60 msgr2=0x7f00dc0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:51.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.412+0000 7f00e140e700 1 -- 172.21.15.193:0/993241063 shutdown_connections 2024-04-25T13:04:51.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.412+0000 7f00e140e700 1 -- 172.21.15.193:0/993241063 wait complete. 2024-04-25T13:04:51.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.413+0000 7f00e140e700 1 Processor -- start 2024-04-25T13:04:51.413 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.413+0000 7f00e140e700 1 -- start start 2024-04-25T13:04:51.413 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.413+0000 7f00e140e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 0x7f00dc198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:51.413 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.413+0000 7f00e140e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc10b830 con 0x7f00dc1015b0 2024-04-25T13:04:51.414 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.414+0000 7f00daffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 0x7f00dc198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:51.414 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.414+0000 7f00daffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 0x7f00dc198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.193:53688/0 (socket says 172.21.15.193:53688) 2024-04-25T13:04:51.414 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.414+0000 7f00daffd700 1 -- 172.21.15.193:0/4206230294 learned_addr learned my addr 172.21.15.193:0/4206230294 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:04:51.414 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.414+0000 7f00daffd700 1 -- 172.21.15.193:0/4206230294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00dc1991d0 con 0x7f00dc1015b0 2024-04-25T13:04:51.414 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.414+0000 7f00daffd700 1 --2- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 0x7f00dc198c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f00cc006010 tx=0x7f00cc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:51.414 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.414+0000 7f00c3fff700 1 -- 172.21.15.193:0/4206230294 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00cc026070 con 0x7f00dc1015b0 2024-04-25T13:04:51.414 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.414+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00dc1994c0 con 0x7f00dc1015b0 2024-04-25T13:04:51.414 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.415+0000 7f00c3fff700 1 -- 172.21.15.193:0/4206230294 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00cc00bba0 con 0x7f00dc1015b0 2024-04-25T13:04:51.414 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.415+0000 7f00c3fff700 1 -- 172.21.15.193:0/4206230294 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00cc003910 con 0x7f00dc1015b0 2024-04-25T13:04:51.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.415+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00dc199980 con 0x7f00dc1015b0 2024-04-25T13:04:51.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.415+0000 7f00c3fff700 1 -- 172.21.15.193:0/4206230294 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f00cc029070 con 0x7f00dc1015b0 2024-04-25T13:04:51.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.416+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00dc102a90 con 0x7f00dc1015b0 2024-04-25T13:04:51.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.416+0000 7f00c3fff700 1 --2- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00c4039650 0x7f00c403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:51.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.416+0000 7f00c3fff700 1 -- 172.21.15.193:0/4206230294 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f00cc01e070 con 0x7f00dc1015b0 2024-04-25T13:04:51.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.416+0000 7f00da7fc700 1 --2- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00c4039650 0x7f00c403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:51.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.416+0000 7f00da7fc700 1 --2- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00c4039650 0x7f00c403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f00d0006fd0 tx=0x7f00d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:51.420 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.421+0000 7f00c3fff700 1 -- 172.21.15.193:0/4206230294 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00cc013070 con 0x7f00dc1015b0 2024-04-25T13:04:51.677 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 podman[32712]: 2024-04-25 13:04:51.461486045 +0000 UTC m=+0.019498550 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:04:51.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 podman[32712]: 2024-04-25 13:04:51.570436701 +0000 UTC m=+0.128449190 container create 6e1c06de1eb50c9e1ba8c5f04ae790b57a3c2a01b7c433a0b4977f1b8d7ea19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:04:51.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.715+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f00dc06af20 con 0x7f00dc1015b0 2024-04-25T13:04:51.714 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.715+0000 7f00c3fff700 1 -- 172.21.15.193:0/4206230294 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f00cc030090 con 0x7f00dc1015b0 2024-04-25T13:04:51.714 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:04:51.714 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","modified":"2024-04-25T13:02:54.770882Z","created":"2024-04-25T13:02:54.770882Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T13:04:51.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.717+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00c4039650 msgr2=0x7f00c403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:51.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.717+0000 7f00e140e700 1 --2- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00c4039650 0x7f00c403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f00d0006fd0 tx=0x7f00d0006e40 comp rx=0 tx=0).stop 2024-04-25T13:04:51.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.717+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 msgr2=0x7f00dc198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:51.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.717+0000 7f00e140e700 1 --2- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 0x7f00dc198c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f00cc006010 tx=0x7f00cc005780 comp rx=0 tx=0).stop 2024-04-25T13:04:51.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.717+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 shutdown_connections 2024-04-25T13:04:51.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.717+0000 7f00e140e700 1 --2- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00c4039650 0x7f00c403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:51.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.717+0000 7f00e140e700 1 --2- 172.21.15.193:0/4206230294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00dc1015b0 0x7f00dc198c90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:51.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.717+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 >> 172.21.15.193:0/4206230294 conn(0x7f00dc0fce60 msgr2=0x7f00dc102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:51.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.718+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 shutdown_connections 2024-04-25T13:04:51.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:51.718+0000 7f00e140e700 1 -- 172.21.15.193:0/4206230294 wait complete. 2024-04-25T13:04:51.717 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T13:04:51.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 podman[32712]: 2024-04-25 13:04:51.79009511 +0000 UTC m=+0.348107599 container init 6e1c06de1eb50c9e1ba8c5f04ae790b57a3c2a01b7c433a0b4977f1b8d7ea19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T13:04:51.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 podman[32712]: 2024-04-25 13:04:51.793399481 +0000 UTC m=+0.351411979 container start 6e1c06de1eb50c9e1ba8c5f04ae790b57a3c2a01b7c433a0b4977f1b8d7ea19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:04:51.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pidfile_write: ignore empty --pid-file 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: load: jerasure load: lrc 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: RocksDB version: 7.9.2 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Git sha 0 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: DB SUMMARY 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: DB Session ID: CXQFEWFCFZY65DIXBV5J 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: CURRENT file: CURRENT 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T13:04:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.error_if_exists: 0 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.create_if_missing: 0 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.paranoid_checks: 1 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.env: 0x559651242cc0 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.info_log: 0x559653031be0 2024-04-25T13:04:51.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T13:04:51.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.statistics: (nil) 2024-04-25T13:04:51.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.use_fsync: 0 2024-04-25T13:04:51.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_log_file_size: 0 2024-04-25T13:04:51.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T13:04:51.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T13:04:51.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T13:04:51.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T13:04:51.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.allow_fallocate: 1 2024-04-25T13:04:51.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.use_direct_reads: 0 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.db_log_dir: 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.wal_dir: 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T13:04:51.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.write_buffer_manager: 0x559652fc50e0 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.unordered_write: 0 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T13:04:51.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.row_cache: None 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.wal_filter: None 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.two_write_queues: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.wal_compression: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.atomic_flush: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.log_readahead_size: 0 2024-04-25T13:04:51.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_background_jobs: 2 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_background_compactions: -1 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_subcompactions: 1 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T13:04:51.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_open_files: -1 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_background_flushes: -1 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Compression algorithms supported: 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: kZSTD supported: 0 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: kXpressCompression supported: 0 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: kZlibCompression supported: 1 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: kSnappyCompression supported: 1 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: kLZ4Compression supported: 1 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: kBZip2Compression supported: 0 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.merge_operator: 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_filter: None 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559653031dc0) 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_type: 0 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_index_type: 0 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_shortening: 1 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: checksum: 4 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: no_block_cache: 0 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache: 0x559652ff71f0 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_options: 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: capacity : 536870912 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_shard_bits : 4 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: strict_capacity_limit : 0 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_compressed: (nil) 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: persistent_cache: (nil) 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size: 4096 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size_deviation: 10 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_restart_interval: 16 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_block_restart_interval: 1 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: metadata_block_size: 4096 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: partition_filters: 0 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: use_delta_encoding: 1 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: filter_policy: bloomfilter 2024-04-25T13:04:51.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: whole_key_filtering: 1 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: verify_compression: 0 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: format_version: 5 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_index_compression: 1 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_align: 0 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: max_auto_readahead_size: 262144 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: prepopulate_block_cache: 0 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: initial_auto_readahead_size: 8192 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression: NoCompression 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T13:04:51.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.num_levels: 7 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:04:51.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T13:04:51.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T13:04:51.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T13:04:51.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.inplace_update_support: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.bloom_locality: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.max_successive_merges: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.ttl: 2592000 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T13:04:51.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.enable_blob_files: false 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.min_blob_size: 0 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 636e31b0-6c09-4352-952f-14775f08f0af 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050291823565, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050291823991, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714050291, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "636e31b0-6c09-4352-952f-14775f08f0af", "db_session_id": "CXQFEWFCFZY65DIXBV5J", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050291824070, "job": 1, "event": "recovery_finished"} 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55965310c000 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: DB pointer 0x5596530f6000 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** DB Stats ** 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:04:52.000 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:04:52.001 INFO:journalctl@ceph.mon.c.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T13:04:52.002 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache BinnedLRUCache@0x559652ff71f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: starting mon.c rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(???) e0 preinit fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: e1 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: legacy client fscid: -1 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-04-25T13:04:52.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: No filesystems configured 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Deploying cephadm binary to smithi193 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Added host smithi193 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3466995133' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3466995133' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T13:04:52.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:52.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi164:172.21.15.164=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Saving service mon spec with placement smithi072:172.21.15.72=a;smithi164:172.21.15.164=b;smithi193:172.21.15.193=c;count:3 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: Deploying daemon mon.c on smithi193 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 ceph-mon[32743]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T13:04:52.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 bash[32712]: 6e1c06de1eb50c9e1ba8c5f04ae790b57a3c2a01b7c433a0b4977f1b8d7ea19a 2024-04-25T13:04:52.009 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:51 smithi193 systemd[1]: Started Ceph mon.c for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:04:53.545 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:04:53.545 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:04:53.788 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:04:57.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:04:57.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: mon.a calling monitor election 2024-04-25T13:04:57.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:57.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: mon.c calling monitor election 2024-04-25T13:04:57.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: monmap epoch 2 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: last_changed 2024-04-25T13:04:51.846123+0000 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: created 2024-04-25T13:02:54.770882+0000 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: min_mon_release 19 (squid) 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: election_strategy: 1 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-04-25T13:04:57.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: fsmap 2024-04-25T13:04:57.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:04:57.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: mgrmap e13: x(active, since 49s) 2024-04-25T13:04:57.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: overall HEALTH_OK 2024-04-25T13:04:57.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:57.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:57.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:57.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:04:57.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: mon.a calling monitor election 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: mon.c calling monitor election 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:57.585 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: monmap epoch 2 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: last_changed 2024-04-25T13:04:51.846123+0000 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: created 2024-04-25T13:02:54.770882+0000 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: min_mon_release 19 (squid) 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: election_strategy: 1 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: fsmap 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:04:57.586 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: mgrmap e13: x(active, since 49s) 2024-04-25T13:04:57.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: overall HEALTH_OK 2024-04-25T13:04:57.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:57.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:57.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:04:57.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:04:57.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:58.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:57 smithi193 ceph-mon[32743]: Deploying daemon mon.b on smithi164 2024-04-25T13:04:58.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:57 smithi193 ceph-mon[32743]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:58.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:58.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:57 smithi072 ceph-mon[29199]: Deploying daemon mon.b on smithi164 2024-04-25T13:04:58.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:57 smithi072 ceph-mon[29199]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:04:58.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:58.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.279+0000 7f1426c79700 1 -- 172.21.15.193:0/2380853625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f140c0035c0 msgr2=0x7f140c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:58.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.279+0000 7f1426c79700 1 --2- 172.21.15.193:0/2380853625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f140c0035c0 0x7f140c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1420101380 tx=0x7f1418009e80 comp rx=0 tx=0).stop 2024-04-25T13:04:58.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.279+0000 7f1426c79700 1 -- 172.21.15.193:0/2380853625 shutdown_connections 2024-04-25T13:04:58.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.279+0000 7f1426c79700 1 --2- 172.21.15.193:0/2380853625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f140c0035c0 0x7f140c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:58.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.279+0000 7f1426c79700 1 -- 172.21.15.193:0/2380853625 >> 172.21.15.193:0/2380853625 conn(0x7f14200fcf80 msgr2=0x7f14200ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:58.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.279+0000 7f1426c79700 1 -- 172.21.15.193:0/2380853625 shutdown_connections 2024-04-25T13:04:58.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.279+0000 7f1426c79700 1 -- 172.21.15.193:0/2380853625 wait complete. 2024-04-25T13:04:58.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.280+0000 7f1426c79700 1 Processor -- start 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.280+0000 7f1426c79700 1 -- start start 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.280+0000 7f1426c79700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 0x7f1420198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.280+0000 7f1426c79700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1420199380 0x7f1420199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.280+0000 7f1426c79700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f14201093a0 con 0x7f1420199380 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.280+0000 7f1426c79700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1420109220 con 0x7f140c0035c0 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.280+0000 7f1424a15700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 0x7f1420198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.280+0000 7f1424a15700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 0x7f1420198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:41032/0 (socket says 172.21.15.193:41032) 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.280+0000 7f1424a15700 1 -- 172.21.15.193:0/2326278810 learned_addr learned my addr 172.21.15.193:0/2326278810 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f141ffff700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1420199380 0x7f1420199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f1424a15700 1 -- 172.21.15.193:0/2326278810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 msgr2=0x7f1420198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T13:04:58.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f1424a15700 1 -- 172.21.15.193:0/2326278810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 msgr2=0x7f1420198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f1424a15700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 0x7f1420198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f1424a15700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 0x7f1420198e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f141ffff700 1 -- 172.21.15.193:0/2326278810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 msgr2=0x7f1420198e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f141ffff700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 0x7f1420198e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f141ffff700 1 -- 172.21.15.193:0/2326278810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14201a1eb0 con 0x7f1420199380 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f141ffff700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1420199380 0x7f1420199800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f141000b4d0 tx=0x7f141000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.281+0000 7f141dffb700 1 -- 172.21.15.193:0/2326278810 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1410011630 con 0x7f1420199380 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.282+0000 7f141dffb700 1 -- 172.21.15.193:0/2326278810 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1410011c80 con 0x7f1420199380 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.282+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14201a20e0 con 0x7f1420199380 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.282+0000 7f141dffb700 1 -- 172.21.15.193:0/2326278810 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1410010d90 con 0x7f1420199380 2024-04-25T13:04:58.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.282+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14201a2670 con 0x7f1420199380 2024-04-25T13:04:58.284 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.282+0000 7f141dffb700 1 -- 172.21.15.193:0/2326278810 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1410018020 con 0x7f1420199380 2024-04-25T13:04:58.284 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.283+0000 7f141dffb700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14000393f0 0x7f140003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:58.284 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.283+0000 7f141dffb700 1 -- 172.21.15.193:0/2326278810 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f141004c1b0 con 0x7f1420199380 2024-04-25T13:04:58.284 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.283+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1420102b90 con 0x7f1420199380 2024-04-25T13:04:58.284 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.283+0000 7f1424a15700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14000393f0 0x7f140003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:58.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.283+0000 7f1424a15700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14000393f0 0x7f140003b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1418009a30 tx=0x7f14180090d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:58.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.289+0000 7f141dffb700 1 -- 172.21.15.193:0/2326278810 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f141001f020 con 0x7f1420199380 2024-04-25T13:04:58.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.571+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f142002df80 con 0x7f1420199380 2024-04-25T13:04:58.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.571+0000 7f141dffb700 1 -- 172.21.15.193:0/2326278810 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f1410018570 con 0x7f1420199380 2024-04-25T13:04:58.571 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:04:58.572 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","modified":"2024-04-25T13:04:51.846123Z","created":"2024-04-25T13:02:54.770882Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.573+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14000393f0 msgr2=0x7f140003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.573+0000 7f1426c79700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14000393f0 0x7f140003b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1418009a30 tx=0x7f14180090d0 comp rx=0 tx=0).stop 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.574+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1420199380 msgr2=0x7f1420199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.574+0000 7f1426c79700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1420199380 0x7f1420199800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f141000b4d0 tx=0x7f141000b890 comp rx=0 tx=0).stop 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.574+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 shutdown_connections 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.574+0000 7f1426c79700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c0035c0 0x7f1420198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.574+0000 7f1426c79700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14000393f0 0x7f140003b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.574+0000 7f1426c79700 1 --2- 172.21.15.193:0/2326278810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1420199380 0x7f1420199800 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.574+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 >> 172.21.15.193:0/2326278810 conn(0x7f14200fcf80 msgr2=0x7f14200fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:58.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.574+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 shutdown_connections 2024-04-25T13:04:58.575 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:04:58.574+0000 7f1426c79700 1 -- 172.21.15.193:0/2326278810 wait complete. 2024-04-25T13:04:58.575 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-04-25T13:04:59.108 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:58 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/2326278810' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:04:59.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:58 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/2326278810' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:05:00.135 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:05:00.136 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:05:00.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:04:59 smithi193 ceph-mon[32743]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:00.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:04:59 smithi072 ceph-mon[29199]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:00.378 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:05:01.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:00 smithi164 systemd[1]: Started Ceph mon.b for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:05:05.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:05:05.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: mon.a calling monitor election 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: mon.c calling monitor election 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: mon.b calling monitor election 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: monmap epoch 3 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: last_changed 2024-04-25T13:05:00.622458+0000 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: created 2024-04-25T13:02:54.770882+0000 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: min_mon_release 19 (squid) 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: election_strategy: 1 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-25T13:05:05.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: fsmap 2024-04-25T13:05:05.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:05:05.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: mgrmap e13: x(active, since 58s) 2024-04-25T13:05:05.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: overall HEALTH_OK 2024-04-25T13:05:05.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:05.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:05.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:05.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: mon.a calling monitor election 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: mon.c calling monitor election 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: mon.b calling monitor election 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: monmap epoch 3 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: last_changed 2024-04-25T13:05:00.622458+0000 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: created 2024-04-25T13:02:54.770882+0000 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: min_mon_release 19 (squid) 2024-04-25T13:05:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: election_strategy: 1 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: fsmap 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: mgrmap e13: x(active, since 58s) 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: overall HEALTH_OK 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:06.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:06.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: mon.a calling monitor election 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: mon.c calling monitor election 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: mon.b calling monitor election 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: monmap epoch 3 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: last_changed 2024-04-25T13:05:00.622458+0000 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: created 2024-04-25T13:02:54.770882+0000 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: min_mon_release 19 (squid) 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: election_strategy: 1 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: fsmap 2024-04-25T13:05:06.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:05:06.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: mgrmap e13: x(active, since 58s) 2024-04-25T13:05:06.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: overall HEALTH_OK 2024-04-25T13:05:06.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:06.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:06.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:06.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:06.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.257+0000 7fdbb85d1700 1 -- 172.21.15.193:0/4113710991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb9c0057d0 msgr2=0x7fdb9c007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:06.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.257+0000 7fdbb85d1700 1 --2- 172.21.15.193:0/4113710991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb9c0057d0 0x7fdb9c007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0101380 tx=0x7fdbac00fe60 comp rx=0 tx=0).stop 2024-04-25T13:05:06.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.257+0000 7fdbb85d1700 1 -- 172.21.15.193:0/4113710991 shutdown_connections 2024-04-25T13:05:06.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.257+0000 7fdbb85d1700 1 --2- 172.21.15.193:0/4113710991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb9c0057d0 0x7fdb9c007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:06.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.257+0000 7fdbb85d1700 1 -- 172.21.15.193:0/4113710991 >> 172.21.15.193:0/4113710991 conn(0x7fdbb00fcf80 msgr2=0x7fdbb00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.257+0000 7fdbb85d1700 1 -- 172.21.15.193:0/4113710991 shutdown_connections 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.257+0000 7fdbb85d1700 1 -- 172.21.15.193:0/4113710991 wait complete. 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb85d1700 1 Processor -- start 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb85d1700 1 -- start start 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb85d1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 0x7fdbb0195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb85d1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbb0196460 0x7fdbb019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb85d1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbb019ad30 0x7fdbb019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb85d1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01082f0 con 0x7fdbb0196460 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb85d1700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb006bb40 con 0x7fdbb019ad30 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb85d1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0108470 con 0x7fdb9c0057d0 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb636d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 0x7fdbb0195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:06.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb636d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 0x7fdbb0195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:36500/0 (socket says 172.21.15.193:36500) 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb636d700 1 -- 172.21.15.193:0/3087490077 learned_addr learned my addr 172.21.15.193:0/3087490077 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb5b6c700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbb0196460 0x7fdbb019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb636d700 1 -- 172.21.15.193:0/3087490077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 msgr2=0x7fdbb0195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb636d700 1 -- 172.21.15.193:0/3087490077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 msgr2=0x7fdbb0195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb636d700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 0x7fdbb0195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb6b6e700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbb019ad30 0x7fdbb019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb636d700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 0x7fdbb0195f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb6b6e700 1 -- 172.21.15.193:0/3087490077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbb019ad30 msgr2=0x7fdbb019b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb6b6e700 1 -- 172.21.15.193:0/3087490077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbb019ad30 msgr2=0x7fdbb019b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb6b6e700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbb019ad30 0x7fdbb019b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb6b6e700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbb019ad30 0x7fdbb019b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:05:06.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb5b6c700 1 -- 172.21.15.193:0/3087490077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 msgr2=0x7fdbb0195f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb5b6c700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 0x7fdbb0195f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb5b6c700 1 -- 172.21.15.193:0/3087490077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbb019ad30 msgr2=0x7fdbb019b1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb5b6c700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbb019ad30 0x7fdbb019b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.258+0000 7fdbb5b6c700 1 -- 172.21.15.193:0/3087490077 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb01a2aa0 con 0x7fdbb0196460 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.259+0000 7fdbb5b6c700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbb0196460 0x7fdbb019a7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b550 tx=0x7fdba000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.259+0000 7fdba77fe700 1 -- 172.21.15.193:0/3087490077 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0004230 con 0x7fdbb0196460 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.259+0000 7fdba77fe700 1 -- 172.21.15.193:0/3087490077 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba00043d0 con 0x7fdbb0196460 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.259+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb01a2cd0 con 0x7fdbb0196460 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.259+0000 7fdba77fe700 1 -- 172.21.15.193:0/3087490077 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0011b90 con 0x7fdbb0196460 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.259+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01a3200 con 0x7fdbb0196460 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.260+0000 7fdba77fe700 1 -- 172.21.15.193:0/3087490077 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdba001a400 con 0x7fdbb0196460 2024-04-25T13:05:06.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.260+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb00640a0 con 0x7fdbb0196460 2024-04-25T13:05:06.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.260+0000 7fdba77fe700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdb94039ae0 0x7fdb9403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:06.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.260+0000 7fdba77fe700 1 -- 172.21.15.193:0/3087490077 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdba004c380 con 0x7fdbb0196460 2024-04-25T13:05:06.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.260+0000 7fdbb636d700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdb94039ae0 0x7fdb9403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:06.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.261+0000 7fdbb636d700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdb94039ae0 0x7fdb9403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00fa10 tx=0x7fdbac005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:06.265 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.266+0000 7fdba77fe700 1 -- 172.21.15.193:0/3087490077 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdba00151c0 con 0x7fdbb0196460 2024-04-25T13:05:06.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.570+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdbb002df80 con 0x7fdbb0196460 2024-04-25T13:05:06.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.571+0000 7fdba77fe700 1 -- 172.21.15.193:0/3087490077 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7fdba001a840 con 0x7fdbb0196460 2024-04-25T13:05:06.570 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T13:05:06.574 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","modified":"2024-04-25T13:05:00.622458Z","created":"2024-04-25T13:02:54.770882Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T13:05:06.575 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdb94039ae0 msgr2=0x7fdb9403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:06.575 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdb94039ae0 0x7fdb9403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00fa10 tx=0x7fdbac005ad0 comp rx=0 tx=0).stop 2024-04-25T13:05:06.575 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbb0196460 msgr2=0x7fdbb019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:06.575 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbb0196460 0x7fdbb019a7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b550 tx=0x7fdba000b910 comp rx=0 tx=0).stop 2024-04-25T13:05:06.575 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 shutdown_connections 2024-04-25T13:05:06.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbb019ad30 0x7fdbb019b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:06.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9c0057d0 0x7fdbb0195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:06.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdb94039ae0 0x7fdb9403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:06.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 --2- 172.21.15.193:0/3087490077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbb0196460 0x7fdbb019a7f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:06.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 >> 172.21.15.193:0/3087490077 conn(0x7fdbb00fcf80 msgr2=0x7fdbb0105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:05:06.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 shutdown_connections 2024-04-25T13:05:06.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:06.573+0000 7fdbb85d1700 1 -- 172.21.15.193:0/3087490077 wait complete. 2024-04-25T13:05:06.576 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-04-25T13:05:06.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:06 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:06.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:06 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/3087490077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:05:06.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:06 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:07.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:06 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:07.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:06 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/3087490077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:05:07.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:06 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:07.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:06 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:07.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:06 smithi164 ceph-mon[32418]: from='client.? 172.21.15.193:0/3087490077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:05:07.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:06 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:05:07.594 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T13:05:07.595 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T13:05:07.846 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:05:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:05:08.275 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:08.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:08.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:08.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:08.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:05:08.308 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:05:08.308 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-25T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:05:08.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:09.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:08 smithi072 ceph-mon[29199]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:05:09.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:08 smithi072 ceph-mon[29199]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:05:09.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:08 smithi072 ceph-mon[29199]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:05:09.227 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:08 smithi072 ceph-mon[29199]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:05:09.227 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:08 smithi072 ceph-mon[29199]: Reconfiguring daemon mon.a on smithi072 2024-04-25T13:05:09.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:08 smithi164 ceph-mon[32418]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:05:09.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:08 smithi164 ceph-mon[32418]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:05:09.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:08 smithi164 ceph-mon[32418]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:05:09.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:08 smithi164 ceph-mon[32418]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:05:09.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:08 smithi164 ceph-mon[32418]: Reconfiguring daemon mon.a on smithi072 2024-04-25T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:08 smithi193 ceph-mon[32743]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:08 smithi193 ceph-mon[32743]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:08 smithi193 ceph-mon[32743]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:08 smithi193 ceph-mon[32743]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:08 smithi193 ceph-mon[32743]: Reconfiguring daemon mon.a on smithi072 2024-04-25T13:05:09.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.888+0000 7efecf9c7700 1 -- 172.21.15.72:0/193416110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec81084e0 msgr2=0x7efec81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:09.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.888+0000 7efecf9c7700 1 --2- 172.21.15.72:0/193416110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec81084e0 0x7efec81088c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009910 tx=0x7efeb8009c20 comp rx=0 tx=0).stop 2024-04-25T13:05:09.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.888+0000 7efecf9c7700 1 -- 172.21.15.72:0/193416110 shutdown_connections 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.888+0000 7efecf9c7700 1 --2- 172.21.15.72:0/193416110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec81084e0 0x7efec81088c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.888+0000 7efecf9c7700 1 -- 172.21.15.72:0/193416110 >> 172.21.15.72:0/193416110 conn(0x7efec80fdf80 msgr2=0x7efec81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.888+0000 7efecf9c7700 1 -- 172.21.15.72:0/193416110 shutdown_connections 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.888+0000 7efecf9c7700 1 -- 172.21.15.72:0/193416110 wait complete. 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.889+0000 7efecf9c7700 1 Processor -- start 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.889+0000 7efecf9c7700 1 -- start start 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.889+0000 7efecf9c7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec81084e0 0x7efec81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.889+0000 7efecf9c7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec8198630 0x7efec819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.889+0000 7efecf9c7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec819cf00 0x7efec819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.889+0000 7efecf9c7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efec810c940 con 0x7efec819cf00 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.889+0000 7efecf9c7700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efec810c7c0 con 0x7efec81084e0 2024-04-25T13:05:09.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.889+0000 7efecf9c7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efec810cac0 con 0x7efec8198630 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.889+0000 7efecdf64700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec819cf00 0x7efec819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecdf64700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec819cf00 0x7efec819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52066/0 (socket says 172.21.15.72:52066) 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecdf64700 1 -- 172.21.15.72:0/2773240638 learned_addr learned my addr 172.21.15.72:0/2773240638 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecd763700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec81084e0 0x7efec81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efeccf62700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec8198630 0x7efec819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecdf64700 1 -- 172.21.15.72:0/2773240638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec8198630 msgr2=0x7efec819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecdf64700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec8198630 0x7efec819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecdf64700 1 -- 172.21.15.72:0/2773240638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec81084e0 msgr2=0x7efec81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecdf64700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec81084e0 0x7efec81980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecdf64700 1 -- 172.21.15.72:0/2773240638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec81a4c70 con 0x7efec819cf00 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecdf64700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec819cf00 0x7efec819d3b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efec000ba50 tx=0x7efec000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efebe7fc700 1 -- 172.21.15.72:0/2773240638 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0019070 con 0x7efec819cf00 2024-04-25T13:05:09.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efebe7fc700 1 -- 172.21.15.72:0/2773240638 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec0014440 con 0x7efec819cf00 2024-04-25T13:05:09.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec81a4ea0 con 0x7efec819cf00 2024-04-25T13:05:09.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efebe7fc700 1 -- 172.21.15.72:0/2773240638 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec000ce60 con 0x7efec819cf00 2024-04-25T13:05:09.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.890+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec81a52f0 con 0x7efec819cf00 2024-04-25T13:05:09.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.891+0000 7efebe7fc700 1 -- 172.21.15.72:0/2773240638 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efec001c070 con 0x7efec819cf00 2024-04-25T13:05:09.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.891+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec80640a0 con 0x7efec819cf00 2024-04-25T13:05:09.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.891+0000 7efebe7fc700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7efeb4039ae0 0x7efeb403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:09.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.891+0000 7efebe7fc700 1 -- 172.21.15.72:0/2773240638 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efec004bdc0 con 0x7efec819cf00 2024-04-25T13:05:09.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.891+0000 7efecd763700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7efeb4039ae0 0x7efeb403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:09.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.892+0000 7efecd763700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7efeb4039ae0 0x7efeb403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006010 tx=0x7efeb801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:09.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:09.896+0000 7efebe7fc700 1 -- 172.21.15.72:0/2773240638 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efec0014910 con 0x7efec819cf00 2024-04-25T13:05:10.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.101+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efec81a5730 con 0x7efec819cf00 2024-04-25T13:05:10.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.101+0000 7efebe7fc700 1 -- 172.21.15.72:0/2773240638 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7efec0007670 con 0x7efec819cf00 2024-04-25T13:05:10.101 INFO:teuthology.orchestra.run.smithi072.stdout:# minimal ceph.conf for d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:05:10.101 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2024-04-25T13:05:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:05:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: mon_host = [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.103+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7efeb4039ae0 msgr2=0x7efeb403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.103+0000 7efecf9c7700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7efeb4039ae0 0x7efeb403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006010 tx=0x7efeb801a040 comp rx=0 tx=0).stop 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec819cf00 msgr2=0x7efec819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec819cf00 0x7efec819d3b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efec000ba50 tx=0x7efec000bd60 comp rx=0 tx=0).stop 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 shutdown_connections 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7efeb4039ae0 0x7efeb403bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efec819cf00 0x7efec819d3b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec81084e0 0x7efec81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 --2- 172.21.15.72:0/2773240638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec8198630 0x7efec819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 >> 172.21.15.72:0/2773240638 conn(0x7efec80fdf80 msgr2=0x7efec8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:05:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 shutdown_connections 2024-04-25T13:05:10.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:10.104+0000 7efecf9c7700 1 -- 172.21.15.72:0/2773240638 wait complete. 2024-04-25T13:05:10.258 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:09 smithi072 ceph-mon[29199]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:10.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:09 smithi164 ceph-mon[32418]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:10.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:10 smithi193 ceph-mon[32743]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:10.852 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T13:05:10.852 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:05:10.852 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:05:10.878 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:05:10.878 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:05:10.941 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T13:05:10.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:05:10.969 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T13:05:10.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:05:11.033 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T13:05:11.033 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:05:11.063 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T13:05:11.064 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:05:11.129 INFO:tasks.cephadm:Adding mgr.x on smithi072 2024-04-25T13:05:11.130 INFO:tasks.cephadm:Adding mgr.y on smithi164 2024-04-25T13:05:11.130 INFO:tasks.cephadm:Adding mgr.z on smithi193 2024-04-25T13:05:11.130 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi072=x;smithi164=y;smithi193=z' 2024-04-25T13:05:11.207 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:11 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2773240638' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:11.207 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:11.207 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:11.207 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:05:11.207 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:05:11.207 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:11.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:11 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2773240638' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:11.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:05:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:05:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:11.402 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:05:11.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:11 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2773240638' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:11.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:11.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:11.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:05:11.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:05:11.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:12.218 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:12 smithi072 ceph-mon[29199]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:05:12.218 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:12 smithi072 ceph-mon[29199]: Reconfiguring daemon mon.b on smithi164 2024-04-25T13:05:12.218 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:12 smithi072 ceph-mon[29199]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:12.219 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:12 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:12.219 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:12 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:12.219 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:12 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:05:12.219 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:12 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:05:12.219 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:12 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:12 smithi193 ceph-mon[32743]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:05:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:12 smithi193 ceph-mon[32743]: Reconfiguring daemon mon.b on smithi164 2024-04-25T13:05:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:12 smithi193 ceph-mon[32743]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:12 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:12 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:12 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:05:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:12 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:05:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:12 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:12.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:12 smithi164 ceph-mon[32418]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:05:12.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:12 smithi164 ceph-mon[32418]: Reconfiguring daemon mon.b on smithi164 2024-04-25T13:05:12.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:12 smithi164 ceph-mon[32418]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:12.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:12 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:12.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:12 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:12.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:12 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:05:12.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:12 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:05:12.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:12 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:13 smithi072 ceph-mon[29199]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:05:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:13 smithi072 ceph-mon[29199]: Reconfiguring daemon mon.c on smithi193 2024-04-25T13:05:13.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:13 smithi164 ceph-mon[32418]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:05:13.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:13 smithi164 ceph-mon[32418]: Reconfiguring daemon mon.c on smithi193 2024-04-25T13:05:13.338 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:13 smithi193 ceph-mon[32743]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:05:13.338 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:13 smithi193 ceph-mon[32743]: Reconfiguring daemon mon.c on smithi193 2024-04-25T13:05:13.812 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.810+0000 7f4445987700 1 -- 172.21.15.193:0/1681097314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440105e50 msgr2=0x7f4440108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:13.812 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.810+0000 7f4445987700 1 --2- 172.21.15.193:0/1681097314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440105e50 0x7f4440108240 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4434009990 tx=0x7f4434009ca0 comp rx=0 tx=0).stop 2024-04-25T13:05:13.812 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.811+0000 7f4445987700 1 -- 172.21.15.193:0/1681097314 shutdown_connections 2024-04-25T13:05:13.812 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.811+0000 7f4445987700 1 --2- 172.21.15.193:0/1681097314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4440108780 0x7f444010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:13.812 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.811+0000 7f4445987700 1 --2- 172.21.15.193:0/1681097314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4440103520 0x7f4440105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.811+0000 7f4445987700 1 --2- 172.21.15.193:0/1681097314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440105e50 0x7f4440108240 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.811+0000 7f4445987700 1 -- 172.21.15.193:0/1681097314 >> 172.21.15.193:0/1681097314 conn(0x7f44400fcf30 msgr2=0x7f44400ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.811+0000 7f4445987700 1 -- 172.21.15.193:0/1681097314 shutdown_connections 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.811+0000 7f4445987700 1 -- 172.21.15.193:0/1681097314 wait complete. 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.811+0000 7f4445987700 1 Processor -- start 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.811+0000 7f4445987700 1 -- start start 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4445987700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440103520 0x7f4440198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4445987700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4440105e50 0x7f4440199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4445987700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4440108780 0x7f44401a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4445987700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f444010e520 con 0x7f4440108780 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4445987700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f444010e3a0 con 0x7f4440105e50 2024-04-25T13:05:13.813 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4445987700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f444010e6a0 con 0x7f4440103520 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440103520 0x7f4440198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440103520 0x7f4440198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:48914/0 (socket says 172.21.15.193:48914) 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443effd700 1 -- 172.21.15.193:0/637668832 learned_addr learned my addr 172.21.15.193:0/637668832 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443f7fe700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4440108780 0x7f44401a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443e7fc700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4440105e50 0x7f4440199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443effd700 1 -- 172.21.15.193:0/637668832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4440105e50 msgr2=0x7f4440199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443effd700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4440105e50 0x7f4440199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443effd700 1 -- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4440108780 msgr2=0x7f44401a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443effd700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4440108780 0x7f44401a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443effd700 1 -- 172.21.15.193:0/637668832 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44401a0860 con 0x7f4440103520 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f443effd700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440103520 0x7f4440198c10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f443000d950 tx=0x7f443000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4444985700 1 -- 172.21.15.193:0/637668832 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430016070 con 0x7f4440103520 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4444985700 1 -- 172.21.15.193:0/637668832 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f443000fd40 con 0x7f4440103520 2024-04-25T13:05:13.814 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44401a0b50 con 0x7f4440103520 2024-04-25T13:05:13.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4444985700 1 -- 172.21.15.193:0/637668832 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443000e3b0 con 0x7f4440103520 2024-04-25T13:05:13.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.812+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44401a1090 con 0x7f4440103520 2024-04-25T13:05:13.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.813+0000 7f4444985700 1 -- 172.21.15.193:0/637668832 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4430019070 con 0x7f4440103520 2024-04-25T13:05:13.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.813+0000 7f4444985700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4428039890 0x7f442803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:13.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.813+0000 7f4444985700 1 -- 172.21.15.193:0/637668832 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f443004c560 con 0x7f4440103520 2024-04-25T13:05:13.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.813+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4440069730 con 0x7f4440103520 2024-04-25T13:05:13.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.814+0000 7f443e7fc700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4428039890 0x7f442803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:13.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.814+0000 7f443e7fc700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4428039890 0x7f442803bd50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f443400b5c0 tx=0x7f4434005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:13.817 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:13.819+0000 7f4444985700 1 -- 172.21.15.193:0/637668832 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f443000f890 con 0x7f4440103520 2024-04-25T13:05:14.028 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.029+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi164=y;smithi193=z", "target": ["mon-mgr", ""]}) -- 0x7f4440062f10 con 0x7f4428039890 2024-04-25T13:05:14.033 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.035+0000 7f4444985700 1 -- 172.21.15.193:0/637668832 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4440062f10 con 0x7f4428039890 2024-04-25T13:05:14.033 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4428039890 msgr2=0x7f442803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4428039890 0x7f442803bd50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f443400b5c0 tx=0x7f4434005fb0 comp rx=0 tx=0).stop 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440103520 msgr2=0x7f4440198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440103520 0x7f4440198c10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f443000d950 tx=0x7f443000dc60 comp rx=0 tx=0).stop 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 shutdown_connections 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4428039890 0x7f442803bd50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4440108780 0x7f44401a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4440103520 0x7f4440198c10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 --2- 172.21.15.193:0/637668832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4440105e50 0x7f4440199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 >> 172.21.15.193:0/637668832 conn(0x7f44400fcf30 msgr2=0x7f4440103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 shutdown_connections 2024-04-25T13:05:14.036 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:05:14.037+0000 7f4445987700 1 -- 172.21.15.193:0/637668832 wait complete. 2024-04-25T13:05:14.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:14 smithi072 ceph-mon[29199]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:14.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:14.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:14.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:14.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:14.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:14.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:14.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:14 smithi164 ceph-mon[32418]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:14.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:14.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:14.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:14.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:14.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:14.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:14 smithi193 ceph-mon[32743]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:14.596 DEBUG:teuthology.orchestra.run.smithi164:mgr.y> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T13:05:14.599 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T13:05:14.601 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T13:05:14.601 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:05:14.602 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:05:14.617 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:05:14.617 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2024-04-25T13:05:14.672 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:05:14.672 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:14.672 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-25T13:05:14.672 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:14.673 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:14.673 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-25 13:04:18.914586681 +0000 2024-04-25T13:05:14.673 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-25 12:58:00.779840485 +0000 2024-04-25T13:05:14.673 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-25 12:58:00.779840485 +0000 2024-04-25T13:05:14.673 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-25 12:58:00.779840485 +0000 2024-04-25T13:05:14.673 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:05:14.736 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-25T13:05:14.736 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-25T13:05:14.737 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.00026603 s, 1.9 MB/s 2024-04-25T13:05:14.738 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:05:14.793 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2024-04-25T13:05:14.847 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:05:14.847 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:14.847 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-04-25T13:05:14.847 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:14.847 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:14.847 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-25 13:04:18.948585961 +0000 2024-04-25T13:05:14.847 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-25 12:58:01.120833096 +0000 2024-04-25T13:05:14.847 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-25 12:58:01.120833096 +0000 2024-04-25T13:05:14.848 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-25 12:58:01.120833096 +0000 2024-04-25T13:05:14.848 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:05:14.910 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-25T13:05:14.910 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-25T13:05:14.911 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000205752 s, 2.5 MB/s 2024-04-25T13:05:14.912 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:05:14.967 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2024-04-25T13:05:15.022 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:05:15.022 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:15.022 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-25T13:05:15.022 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:15.022 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:15.022 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-25 13:04:18.980585282 +0000 2024-04-25T13:05:15.022 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-25 12:58:01.459825756 +0000 2024-04-25T13:05:15.022 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-25 12:58:01.459825756 +0000 2024-04-25T13:05:15.022 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-25 12:58:01.459825756 +0000 2024-04-25T13:05:15.023 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:05:15.084 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-25T13:05:15.084 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-25T13:05:15.084 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000192763 s, 2.7 MB/s 2024-04-25T13:05:15.085 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:05:15.141 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2024-04-25T13:05:15.196 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:05:15.196 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:15.196 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-25T13:05:15.196 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:15.196 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:15.196 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-25 13:04:19.012584604 +0000 2024-04-25T13:05:15.197 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-25 12:58:01.810818157 +0000 2024-04-25T13:05:15.197 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-25 12:58:01.810818157 +0000 2024-04-25T13:05:15.197 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-25 12:58:01.810818157 +0000 2024-04-25T13:05:15.197 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:05:15.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:15 smithi072 ceph-mon[29199]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi164=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:05:15.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:15 smithi072 ceph-mon[29199]: Saving service mgr spec with placement smithi072=x;smithi164=y;smithi193=z;count:3 2024-04-25T13:05:15.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:15 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:15.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:15 smithi072 ceph-mon[29199]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:05:15.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:15 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:05:15.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:15 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:05:15.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:15 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:15.258 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:15 smithi072 ceph-mon[29199]: Reconfiguring daemon mgr.x on smithi072 2024-04-25T13:05:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-25T13:05:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-25T13:05:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000190307 s, 2.7 MB/s 2024-04-25T13:05:15.260 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:05:15.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:15 smithi164 ceph-mon[32418]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi164=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:05:15.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:15 smithi164 ceph-mon[32418]: Saving service mgr spec with placement smithi072=x;smithi164=y;smithi193=z;count:3 2024-04-25T13:05:15.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:15 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:15.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:15 smithi164 ceph-mon[32418]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:05:15.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:15 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:05:15.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:15 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:05:15.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:15 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:15.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:15 smithi164 ceph-mon[32418]: Reconfiguring daemon mgr.x on smithi072 2024-04-25T13:05:15.316 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T13:05:15.316 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:05:15.334 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:05:15.335 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-04-25T13:05:15.392 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:05:15.392 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:15.392 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T13:05:15.392 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:15.392 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:15.392 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-25 13:05:00.055832151 +0000 2024-04-25T13:05:15.392 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-25 12:58:02.191174026 +0000 2024-04-25T13:05:15.392 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-25 12:58:02.191174026 +0000 2024-04-25T13:05:15.393 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-25 12:58:02.191174026 +0000 2024-04-25T13:05:15.393 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:05:15.458 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-25T13:05:15.458 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-25T13:05:15.458 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000285892 s, 1.8 MB/s 2024-04-25T13:05:15.459 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:05:15.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:15 smithi193 ceph-mon[32743]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi164=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:05:15.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:15 smithi193 ceph-mon[32743]: Saving service mgr spec with placement smithi072=x;smithi164=y;smithi193=z;count:3 2024-04-25T13:05:15.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:15 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:15.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:15 smithi193 ceph-mon[32743]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:05:15.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:15 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:05:15.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:15 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:05:15.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:15 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:15.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:15 smithi193 ceph-mon[32743]: Reconfiguring daemon mgr.x on smithi072 2024-04-25T13:05:15.517 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-04-25T13:05:15.573 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:05:15.573 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:15.573 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T13:05:15.573 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:15.573 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:15.573 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-25 13:05:00.055832151 +0000 2024-04-25T13:05:15.573 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-25 12:58:02.557167651 +0000 2024-04-25T13:05:15.574 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-25 12:58:02.557167651 +0000 2024-04-25T13:05:15.574 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-25 12:58:02.557167651 +0000 2024-04-25T13:05:15.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:05:15.639 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-25T13:05:15.639 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-25T13:05:15.639 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000231838 s, 2.2 MB/s 2024-04-25T13:05:15.640 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:05:15.698 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-04-25T13:05:15.755 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:05:15.755 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:15.755 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T13:05:15.756 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:15.756 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:15.756 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-25 13:05:00.056832135 +0000 2024-04-25T13:05:15.756 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-25 12:58:02.917161380 +0000 2024-04-25T13:05:15.756 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-25 12:58:02.917161380 +0000 2024-04-25T13:05:15.756 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-25 12:58:02.917161380 +0000 2024-04-25T13:05:15.756 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:05:15.820 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-25T13:05:15.820 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-25T13:05:15.820 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000203376 s, 2.5 MB/s 2024-04-25T13:05:15.821 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:05:15.878 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-04-25T13:05:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:05:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:15.934 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T13:05:15.934 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:15.934 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:15.935 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-25 13:05:00.056832135 +0000 2024-04-25T13:05:15.935 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-25 12:58:03.269155249 +0000 2024-04-25T13:05:15.935 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-25 12:58:03.269155249 +0000 2024-04-25T13:05:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-25 12:58:03.269155249 +0000 2024-04-25T13:05:15.935 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:05:15.998 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-25T13:05:15.998 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-25T13:05:15.998 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000274802 s, 1.9 MB/s 2024-04-25T13:05:15.999 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:05:16.055 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T13:05:16.055 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:05:16.070 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:05:16.070 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-04-25T13:05:16.125 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:05:16.125 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:16.125 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T13:05:16.125 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:16.125 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:16.126 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 13:04:50.933039331 +0000 2024-04-25T13:05:16.126 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 12:58:05.995600053 +0000 2024-04-25T13:05:16.126 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 12:58:05.995600053 +0000 2024-04-25T13:05:16.126 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 12:58:05.995600053 +0000 2024-04-25T13:05:16.126 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:05:16.188 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T13:05:16.188 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T13:05:16.189 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000251596 s, 2.0 MB/s 2024-04-25T13:05:16.190 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:05:16.245 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-04-25T13:05:16.301 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:05:16.301 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:16.301 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T13:05:16.301 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:16.301 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:16.301 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 13:04:50.933039331 +0000 2024-04-25T13:05:16.301 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 12:58:06.330594430 +0000 2024-04-25T13:05:16.301 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 12:58:06.330594430 +0000 2024-04-25T13:05:16.301 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 12:58:06.330594430 +0000 2024-04-25T13:05:16.302 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:05:16.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:05:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:05:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:05:16.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:16.352 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:05:16.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:16.370 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T13:05:16.370 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T13:05:16.370 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000185087 s, 2.8 MB/s 2024-04-25T13:05:16.371 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:05:16.427 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-04-25T13:05:16.483 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:05:16.483 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:16.483 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T13:05:16.483 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:16.483 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:16.483 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 13:04:50.933039331 +0000 2024-04-25T13:05:16.483 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 12:58:06.668588756 +0000 2024-04-25T13:05:16.484 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 12:58:06.668588756 +0000 2024-04-25T13:05:16.484 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 12:58:06.668588756 +0000 2024-04-25T13:05:16.484 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:05:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:05:16.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:16.548 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T13:05:16.548 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T13:05:16.548 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000189741 s, 2.7 MB/s 2024-04-25T13:05:16.549 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:05:16.605 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-04-25T13:05:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:05:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:05:16.661 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T13:05:16.661 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:05:16.662 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:05:16.662 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 13:04:50.934039315 +0000 2024-04-25T13:05:16.662 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 12:58:07.018582880 +0000 2024-04-25T13:05:16.662 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 12:58:07.018582880 +0000 2024-04-25T13:05:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 12:58:07.018582880 +0000 2024-04-25T13:05:16.662 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:05:16.725 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T13:05:16.725 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T13:05:16.725 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000171508 s, 3.0 MB/s 2024-04-25T13:05:16.727 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:05:16.783 INFO:tasks.cephadm:Deploying osd.0 on smithi072 with /dev/vg_nvme/lv_4... 2024-04-25T13:05:16.783 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:05:17.024 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:05:17.146 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:17 smithi072 ceph-mon[29199]: Deploying daemon mgr.z on smithi193 2024-04-25T13:05:17.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:17 smithi164 ceph-mon[32418]: Deploying daemon mgr.z on smithi193 2024-04-25T13:05:17.335 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:17 smithi193 ceph-mon[32743]: Deploying daemon mgr.z on smithi193 2024-04-25T13:05:17.980 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:17 smithi193 systemd[1]: Starting Ceph mgr.z for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:05:18.274 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:18 smithi193 ceph-mon[32743]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:18.274 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:17 smithi193 podman[34625]: 2024-04-25 13:05:17.990202525 +0000 UTC m=+0.111633310 container create d1e176b2ff2e9327e63ce62734327e23b46cbf4cf23736713464e32be7f4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T13:05:18.275 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:17 smithi193 podman[34625]: 2024-04-25 13:05:17.896846035 +0000 UTC m=+0.018276827 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:05:18.275 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:18 smithi193 podman[34625]: 2024-04-25 13:05:18.175876781 +0000 UTC m=+0.297307564 container init d1e176b2ff2e9327e63ce62734327e23b46cbf4cf23736713464e32be7f4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:05:18.275 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:18 smithi193 podman[34625]: 2024-04-25 13:05:18.178377499 +0000 UTC m=+0.299808282 container start d1e176b2ff2e9327e63ce62734327e23b46cbf4cf23736713464e32be7f4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T13:05:18.275 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:18 smithi193 ceph-mgr[34644]: -- 172.21.15.193:0/70970013 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556cc5dcc5a0 con 0x556cc5da0c00 2024-04-25T13:05:18.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:18 smithi164 ceph-mon[32418]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:18.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:18 smithi072 ceph-mon[29199]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:18.614 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:18 smithi193 bash[34625]: d1e176b2ff2e9327e63ce62734327e23b46cbf4cf23736713464e32be7f4e61e 2024-04-25T13:05:18.615 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:18 smithi193 systemd[1]: Started Ceph mgr.z for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:05:18.615 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:18.328+0000 7f4931c3d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:05:18.615 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:18.514+0000 7f4931c3d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:05:18.615 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:18.615+0000 7f4931c3d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:05:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:05:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:05:18.942 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-04-25T13:05:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-04-25T13:05:18.942 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113336 s, 925 MB/s 2024-04-25T13:05:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-04-25T13:05:19.230 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:18.926+0000 7f4931c3d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:05:19.230 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:19.010+0000 7f4931c3d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:05:19.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:19 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:19.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:19 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:19.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:19 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:19.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:19 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:05:19.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:19 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:05:19.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:19 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:05:19.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:19 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:19.521 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:19 smithi072 ceph-mon[29199]: Deploying daemon mgr.y on smithi164 2024-04-25T13:05:19.521 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:19 smithi072 ceph-mon[29199]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:19.547 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_4 2024-04-25T13:05:19.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:19.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:19.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:19.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:05:19.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:05:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:05:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-mon[32743]: Deploying daemon mgr.y on smithi164 2024-04-25T13:05:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:19 smithi193 ceph-mon[32743]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:19.782 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:05:19.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:19 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:19.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:19 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:19.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:19 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:19.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:19 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:05:19.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:19 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:05:19.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:19 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:05:19.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:19 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:19.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:19 smithi164 ceph-mon[32418]: Deploying daemon mgr.y on smithi164 2024-04-25T13:05:19.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:19 smithi164 ceph-mon[32418]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:20.480 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:20.213+0000 7f4931c3d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:05:20.480 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:20.379+0000 7f4931c3d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:05:20.549 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:20 smithi164 systemd[1]: Starting Ceph mgr.y for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:05:20.800 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:20 smithi164 podman[33823]: 2024-04-25 13:05:20.453483158 +0000 UTC m=+0.017766677 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:05:20.801 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:20 smithi164 podman[33823]: 2024-04-25 13:05:20.555548681 +0000 UTC m=+0.119832194 container create 69b024243b70d41890028e011604025395142883ba403ac686e84d067dd869ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:05:20.801 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:20 smithi164 podman[33823]: 2024-04-25 13:05:20.774691292 +0000 UTC m=+0.338974808 container init 69b024243b70d41890028e011604025395142883ba403ac686e84d067dd869ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T13:05:20.801 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:20 smithi164 podman[33823]: 2024-04-25 13:05:20.777196668 +0000 UTC m=+0.341480189 container start 69b024243b70d41890028e011604025395142883ba403ac686e84d067dd869ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:05:20.801 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:20 smithi164 ceph-mgr[33842]: -- 172.21.15.164:0/95126943 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622f26985a0 con 0x5622f266cc00 2024-04-25T13:05:20.837 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:20.551+0000 7f4931c3d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:05:20.837 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:20.838+0000 7f4931c3d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:05:21.055 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:20 smithi164 bash[33823]: 69b024243b70d41890028e011604025395142883ba403ac686e84d067dd869ff 2024-04-25T13:05:21.056 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:20 smithi164 systemd[1]: Started Ceph mgr.y for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:05:21.056 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:20 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:20.939+0000 7f2f1f356200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:05:21.133 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:20.929+0000 7f4931c3d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:05:21.133 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:21.134+0000 7f4931c3d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:05:21.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 -- 172.21.15.72:0/743479292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46741035d0 msgr2=0x7f46741059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 --2- 172.21.15.72:0/743479292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46741035d0 0x7f46741059c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4664009990 tx=0x7f4664009ca0 comp rx=0 tx=0).stop 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 -- 172.21.15.72:0/743479292 shutdown_connections 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 --2- 172.21.15.72:0/743479292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4674108830 0x7f467410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 --2- 172.21.15.72:0/743479292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46741035d0 0x7f46741059c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 --2- 172.21.15.72:0/743479292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4674105f00 0x7f46741082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 -- 172.21.15.72:0/743479292 >> 172.21.15.72:0/743479292 conn(0x7f46740fcf40 msgr2=0x7f46740ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 -- 172.21.15.72:0/743479292 shutdown_connections 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 -- 172.21.15.72:0/743479292 wait complete. 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.319+0000 7f4678f30700 1 Processor -- start 2024-04-25T13:05:21.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4678f30700 1 -- start start 2024-04-25T13:05:21.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4678f30700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46741035d0 0x7f4674198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:21.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4678f30700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4674105f00 0x7f46741991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:21.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4678f30700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4674108830 0x7f46741a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4678f30700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f467410e500 con 0x7f4674105f00 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4678f30700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f467410e380 con 0x7f46741035d0 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4678f30700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f467410e680 con 0x7f4674108830 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4669d9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4674105f00 0x7f46741991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4669d9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4674105f00 0x7f46741991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39706/0 (socket says 172.21.15.72:39706) 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4669d9b700 1 -- 172.21.15.72:0/855816654 learned_addr learned my addr 172.21.15.72:0/855816654 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f467259c700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46741035d0 0x7f4674198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4672d9d700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4674108830 0x7f46741a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4669d9b700 1 -- 172.21.15.72:0/855816654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4674108830 msgr2=0x7f46741a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4669d9b700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4674108830 0x7f46741a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4669d9b700 1 -- 172.21.15.72:0/855816654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46741035d0 msgr2=0x7f4674198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:21.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4669d9b700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46741035d0 0x7f4674198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4669d9b700 1 -- 172.21.15.72:0/855816654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46741a08c0 con 0x7f4674105f00 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4672d9d700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4674108830 0x7f46741a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.320+0000 7f4669d9b700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4674105f00 0x7f46741991b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f465c00d980 tx=0x7f465c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.321+0000 7f466bfff700 1 -- 172.21.15.72:0/855816654 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c016070 con 0x7f4674105f00 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.321+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46741a0bb0 con 0x7f4674105f00 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.321+0000 7f466bfff700 1 -- 172.21.15.72:0/855816654 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c004830 con 0x7f4674105f00 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.321+0000 7f466bfff700 1 -- 172.21.15.72:0/855816654 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0054c0 con 0x7f4674105f00 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.321+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46741a1140 con 0x7f4674105f00 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.322+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4674069730 con 0x7f4674105f00 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.322+0000 7f466bfff700 1 -- 172.21.15.72:0/855816654 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f465c00b750 con 0x7f4674105f00 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.322+0000 7f466bfff700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4660039890 0x7f466003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.322+0000 7f466bfff700 1 -- 172.21.15.72:0/855816654 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f465c04c880 con 0x7f4674105f00 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.322+0000 7f467259c700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4660039890 0x7f466003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:21.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.322+0000 7f467259c700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4660039890 0x7f466003bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f466400b5c0 tx=0x7f4664005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:21.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.327+0000 7f466bfff700 1 -- 172.21.15.72:0/855816654 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f465c01f020 con 0x7f4674105f00 2024-04-25T13:05:21.480 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:21.334+0000 7f4931c3d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:05:21.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:21.535+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4674062e80 con 0x7f4660039890 2024-04-25T13:05:21.555 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:21.126+0000 7f2f1f356200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:05:21.555 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:21.230+0000 7f2f1f356200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:05:21.922 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:21.557+0000 7f2f1f356200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:05:21.922 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:21.649+0000 7f2f1f356200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:05:22.230 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:21.969+0000 7f4931c3d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:05:22.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-mon[32743]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:22.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:22.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:05:22.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:05:22.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:22.251 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.251 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.251 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.251 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.251 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:21 smithi072 ceph-mon[29199]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:22.251 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:22.252 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:05:22.252 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:05:22.252 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:22.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:22.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-mon[32418]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:22.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:22.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:05:22.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:05:22.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:22.707 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:22.401+0000 7f4931c3d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:05:22.707 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:22.502+0000 7f4931c3d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:05:22.707 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:22.708+0000 7f4931c3d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:05:23.256 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:22 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:22.918+0000 7f2f1f356200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:05:23.256 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:23.085+0000 7f2f1f356200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:05:23.256 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:23.257+0000 7f2f1f356200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:05:23.256 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:23.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.291 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:05:23.291 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.291 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.291 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.291 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.291 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.291 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:23.291 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:23.292 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.292 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:23.292 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:23.292 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:23.292 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.480 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:23.231+0000 7f4931c3d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:05:23.480 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:23.321+0000 7f4931c3d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:05:23.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:05:23.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:23.805 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:23.543+0000 7f2f1f356200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:05:23.805 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:23.636+0000 7f2f1f356200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:05:23.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:23.560+0000 7f4931c3d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:05:23.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:23.693+0000 7f4931c3d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:05:23.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:23.787+0000 7f4931c3d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:05:23.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:23.903+0000 7f4931c3d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:05:24.105 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:23 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:23.839+0000 7f2f1f356200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:05:24.105 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:24 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:24.029+0000 7f2f1f356200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:05:24.221 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:24 smithi193 ceph-mon[32743]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:24.436 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:24 smithi072 ceph-mon[29199]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:24.480 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:24.222+0000 7f4931c3d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:05:24.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:24 smithi164 ceph-mon[32418]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:25.046 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:24 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:24.629+0000 7f2f1f356200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:05:25.130 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:24.816+0000 7f4931c3d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:05:25.130 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:05:25.028+0000 7f4931c3d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:05:25.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:25.142+0000 7f466bfff700 1 -- 172.21.15.72:0/855816654 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f465c003680 con 0x7f4674105f00 2024-04-25T13:05:25.305 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:25.047+0000 7f2f1f356200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:25.144+0000 7f2f1f356200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2930840644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86128e0d-c8b1-4771-a222-d42316fe3351"}]: dispatch 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2930840644' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86128e0d-c8b1-4771-a222-d42316fe3351"}]': finished 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/759098646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: Standby manager daemon z started 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:05:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:05:25.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:05:25.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2930840644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86128e0d-c8b1-4771-a222-d42316fe3351"}]: dispatch 2024-04-25T13:05:25.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2930840644' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86128e0d-c8b1-4771-a222-d42316fe3351"}]': finished 2024-04-25T13:05:25.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:05:25.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:25.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/759098646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:05:25.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: Standby manager daemon z started 2024-04-25T13:05:25.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:05:25.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:05:25.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:05:25.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:25 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2930840644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86128e0d-c8b1-4771-a222-d42316fe3351"}]: dispatch 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2930840644' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86128e0d-c8b1-4771-a222-d42316fe3351"}]': finished 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/759098646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: Standby manager daemon z started 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:05:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:25 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.193:0/11636707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:05:25.805 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:25.342+0000 7f2f1f356200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:05:26.139 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:25.868+0000 7f2f1f356200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:05:26.140 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:25 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:25.959+0000 7f2f1f356200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:05:26.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:26 smithi164 ceph-mon[32418]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:26.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:26 smithi164 ceph-mon[32418]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T13:05:26.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:05:26.424 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:26 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:26.200+0000 7f2f1f356200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:05:26.425 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:26 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:26.331+0000 7f2f1f356200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:05:26.425 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:26 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:26.425+0000 7f2f1f356200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:05:26.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:26 smithi193 ceph-mon[32743]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:26.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:26 smithi193 ceph-mon[32743]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T13:05:26.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:05:26.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:26 smithi072 ceph-mon[29199]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:26.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:26 smithi072 ceph-mon[29199]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T13:05:26.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:05:26.805 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:26 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:26.540+0000 7f2f1f356200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:05:27.305 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:26 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:26.843+0000 7f2f1f356200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:05:27.805 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:27 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:27.436+0000 7f2f1f356200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:05:27.806 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:05:27 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:05:27.648+0000 7f2f1f356200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:05:28.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:28.151+0000 7f466bfff700 1 -- 172.21.15.72:0/855816654 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f465c003ba0 con 0x7f4674105f00 2024-04-25T13:05:28.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:28 smithi072 ceph-mon[29199]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:28.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:28 smithi072 ceph-mon[29199]: Standby manager daemon y started 2024-04-25T13:05:28.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:28 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:05:28.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:28 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:05:28.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:28 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:05:28.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:28 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:05:28.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:28 smithi164 ceph-mon[32418]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:28.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:28 smithi164 ceph-mon[32418]: Standby manager daemon y started 2024-04-25T13:05:28.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:28 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:05:28.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:28 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:05:28.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:28 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:05:28.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:28 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:05:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:28 smithi193 ceph-mon[32743]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:28 smithi193 ceph-mon[32743]: Standby manager daemon y started 2024-04-25T13:05:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:28 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:05:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:28 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:05:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:28 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:05:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:28 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.164:0/1319619398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:05:29.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:29 smithi193 ceph-mon[32743]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T13:05:29.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:29 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:05:29.488 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:29 smithi072 ceph-mon[29199]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T13:05:29.488 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:29 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:05:29.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:29 smithi164 ceph-mon[32418]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T13:05:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:29 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:05:30.427 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:30 smithi072 ceph-mon[29199]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:30.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:30 smithi193 ceph-mon[32743]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:30.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:30 smithi164 ceph-mon[32418]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:32.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:32 smithi072 ceph-mon[29199]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:32.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:05:32.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:32.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:32 smithi193 ceph-mon[32743]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:32.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:05:32.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:32.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:32 smithi164 ceph-mon[32418]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:05:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:33.412 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:33 smithi072 ceph-mon[29199]: Deploying daemon osd.0 on smithi072 2024-04-25T13:05:33.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:33 smithi193 ceph-mon[32743]: Deploying daemon osd.0 on smithi072 2024-04-25T13:05:33.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:33 smithi164 ceph-mon[32418]: Deploying daemon osd.0 on smithi072 2024-04-25T13:05:34.282 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:34 smithi072 ceph-mon[29199]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:34.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:34 smithi193 ceph-mon[32743]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:34.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:34 smithi164 ceph-mon[32418]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:36.266 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:36 smithi072 ceph-mon[29199]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:36 smithi193 ceph-mon[32743]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:36.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:36 smithi164 ceph-mon[32418]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:38.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:37 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:38.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:37 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:38.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:37 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:38.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:37 smithi072 ceph-mon[29199]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:38.234 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:37 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:38.234 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:37 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:38.234 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:37 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:38.234 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:37 smithi193 ceph-mon[32743]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:38.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:37 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:38.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:37 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:38.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:37 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:38.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:37 smithi164 ceph-mon[32418]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:40.409 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:40 smithi072 ceph-mon[29199]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:40.409 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:40 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:40.409 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:40 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:40.409 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:40 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:40.409 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:40 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:40.410 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:40 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:40.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:40 smithi193 ceph-mon[32743]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:40.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:40 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:40.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:40 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:40.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:40 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:40.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:40 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:40.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:40 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:40.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.492+0000 7f466bfff700 1 -- 172.21.15.72:0/855816654 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4674062e80 con 0x7f4660039890 2024-04-25T13:05:40.492 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 0 on host 'smithi072' 2024-04-25T13:05:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.494+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4660039890 msgr2=0x7f466003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.494+0000 7f4678f30700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4660039890 0x7f466003bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f466400b5c0 tx=0x7f4664005fb0 comp rx=0 tx=0).stop 2024-04-25T13:05:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.494+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4674105f00 msgr2=0x7f46741991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.494+0000 7f4678f30700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4674105f00 0x7f46741991b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f465c00d980 tx=0x7f465c00dc90 comp rx=0 tx=0).stop 2024-04-25T13:05:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.495+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 shutdown_connections 2024-04-25T13:05:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.495+0000 7f4678f30700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4674108830 0x7f46741a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.495+0000 7f4678f30700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46741035d0 0x7f4674198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.495+0000 7f4678f30700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4660039890 0x7f466003bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.495+0000 7f4678f30700 1 --2- 172.21.15.72:0/855816654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4674105f00 0x7f46741991b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.495+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 >> 172.21.15.72:0/855816654 conn(0x7f46740fcf40 msgr2=0x7f46741054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:05:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.495+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 shutdown_connections 2024-04-25T13:05:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:05:40.495+0000 7f4678f30700 1 -- 172.21.15.72:0/855816654 wait complete. 2024-04-25T13:05:40.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:40 smithi164 ceph-mon[32418]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:40.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:40 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:40.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:40 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:40.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:40 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:40.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:40 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:40.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:40 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:41.107 DEBUG:teuthology.orchestra.run.smithi072:osd.0> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T13:05:41.110 INFO:tasks.cephadm:Deploying osd.1 on smithi164 with /dev/vg_nvme/lv_4... 2024-04-25T13:05:41.110 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:05:41.355 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:05:41.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:41.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:41.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:41.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:41 smithi072 ceph-mon[29199]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:05:41.477 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:41.478 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:41.478 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:41.478 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:41 smithi164 ceph-mon[32418]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:05:41.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:41.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:41.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:41.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:41 smithi193 ceph-mon[32743]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:05:42.197 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:42 smithi072 ceph-mon[29199]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:42.197 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:42 smithi072 ceph-mon[29199]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:05:42.198 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:42 smithi072 ceph-mon[29199]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:05:42.198 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:42 smithi072 ceph-mon[29199]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:05:42.198 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:42 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:42.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:42 smithi193 ceph-mon[32743]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:42.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:42 smithi193 ceph-mon[32743]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:05:42.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:42 smithi193 ceph-mon[32743]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:05:42.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:42 smithi193 ceph-mon[32743]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:05:42.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:42 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:42.506 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:42 smithi164 ceph-mon[32418]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:42.506 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:42 smithi164 ceph-mon[32418]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:05:42.506 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:42 smithi164 ceph-mon[32418]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:05:42.506 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:42 smithi164 ceph-mon[32418]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:05:42.506 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:42 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:43.310 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:05:43.311 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:05:43.311 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-25T13:05:43.311 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-25T13:05:43.311 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123164 s, 851 MB/s 2024-04-25T13:05:43.311 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-25T13:05:43.685 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:43 smithi164 ceph-mon[32418]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:05:43.685 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:43 smithi164 ceph-mon[32418]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:05:43.685 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:43.685 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:43.685 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:43 smithi164 ceph-mon[32418]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:43.686 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:43 smithi164 ceph-mon[32418]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' 2024-04-25T13:05:43.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:43 smithi193 ceph-mon[32743]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:05:43.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:43 smithi193 ceph-mon[32743]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:05:43.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:43.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:43.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:43 smithi193 ceph-mon[32743]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:43.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:43 smithi193 ceph-mon[32743]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' 2024-04-25T13:05:43.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:43 smithi072 ceph-mon[29199]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:05:43.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:43 smithi072 ceph-mon[29199]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:05:43.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:43.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:43.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:43 smithi072 ceph-mon[29199]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:43.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:43 smithi072 ceph-mon[29199]: from='osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660]' entity='osd.0' 2024-04-25T13:05:43.773 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:05:43 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0[44263]: 2024-04-25T13:05:43.389+0000 7fd9da1de700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T13:05:43.773 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:05:43 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0[44263]: 2024-04-25T13:05:43.539+0000 7fd9d1fc9700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:05:43.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2024-04-25T13:05:44.221 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:05:44.621 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi072 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi072 to 17088M 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:44.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.623 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:44.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:05:44.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:05:44.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:44.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi072 2024-04-25T13:05:44.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:05:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi072 to 17088M 2024-04-25T13:05:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi072 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi072 to 17088M 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:44.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:05:44.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:45.727 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:45 smithi164 ceph-mon[32418]: osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] boot 2024-04-25T13:05:45.728 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:45 smithi164 ceph-mon[32418]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:05:45.728 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:45.728 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:45 smithi164 ceph-mon[32418]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:45.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:45 smithi193 ceph-mon[32743]: osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] boot 2024-04-25T13:05:45.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:45 smithi193 ceph-mon[32743]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:05:45.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:45.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:45 smithi193 ceph-mon[32743]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:45.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.733+0000 7effd0f5e700 1 -- 172.21.15.164:0/2075787820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 msgr2=0x7effcc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:45.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.733+0000 7effd0f5e700 1 --2- 172.21.15.164:0/2075787820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 0x7effcc101dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effbc0099c0 tx=0x7effbc009cd0 comp rx=0 tx=0).stop 2024-04-25T13:05:45.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.733+0000 7effd0f5e700 1 -- 172.21.15.164:0/2075787820 shutdown_connections 2024-04-25T13:05:45.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.733+0000 7effd0f5e700 1 --2- 172.21.15.164:0/2075787820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effcc103540 0x7effcc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:45.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.733+0000 7effd0f5e700 1 --2- 172.21.15.164:0/2075787820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 0x7effcc101dc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:45.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.733+0000 7effd0f5e700 1 --2- 172.21.15.164:0/2075787820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effcc102ba0 0x7effcc103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:45.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.733+0000 7effd0f5e700 1 -- 172.21.15.164:0/2075787820 >> 172.21.15.164:0/2075787820 conn(0x7effcc0fcf40 msgr2=0x7effcc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:05:45.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.733+0000 7effd0f5e700 1 -- 172.21.15.164:0/2075787820 shutdown_connections 2024-04-25T13:05:45.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.733+0000 7effd0f5e700 1 -- 172.21.15.164:0/2075787820 wait complete. 2024-04-25T13:05:45.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effd0f5e700 1 Processor -- start 2024-04-25T13:05:45.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effd0f5e700 1 -- start start 2024-04-25T13:05:45.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effd0f5e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 0x7effcc198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:45.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effd0f5e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effcc102ba0 0x7effcc199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effd0f5e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effcc103540 0x7effcc1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effd0f5e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10d4e0 con 0x7effcc103540 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effd0f5e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10d360 con 0x7effcc1019a0 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effd0f5e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10d660 con 0x7effcc102ba0 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effca59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 0x7effcc198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effca59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 0x7effcc198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49178/0 (socket says 172.21.15.164:49178) 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effca59c700 1 -- 172.21.15.164:0/3498112491 learned_addr learned my addr 172.21.15.164:0/3498112491 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effc9d9b700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effcc102ba0 0x7effcc199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effca59c700 1 -- 172.21.15.164:0/3498112491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effcc102ba0 msgr2=0x7effcc199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effcad9d700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effcc103540 0x7effcc1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effca59c700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effcc102ba0 0x7effcc199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effca59c700 1 -- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effcc103540 msgr2=0x7effcc1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:05:45.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effca59c700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effcc103540 0x7effcc1a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:05:45.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effca59c700 1 -- 172.21.15.164:0/3498112491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc1a0940 con 0x7effcc1019a0 2024-04-25T13:05:45.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effca59c700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 0x7effcc198d40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effbc0098e0 tx=0x7effbc00fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:45.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.734+0000 7effbb7fe700 1 -- 172.21.15.164:0/3498112491 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc027070 con 0x7effcc1019a0 2024-04-25T13:05:45.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.735+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effcc1a0bd0 con 0x7effcc1019a0 2024-04-25T13:05:45.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.735+0000 7effbb7fe700 1 -- 172.21.15.164:0/3498112491 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effbc022920 con 0x7effcc1019a0 2024-04-25T13:05:45.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.735+0000 7effbb7fe700 1 -- 172.21.15.164:0/3498112491 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc009e60 con 0x7effcc1019a0 2024-04-25T13:05:45.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.735+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effcc1a1080 con 0x7effcc1019a0 2024-04-25T13:05:45.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.735+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc069730 con 0x7effcc1019a0 2024-04-25T13:05:45.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.737+0000 7effbb7fe700 1 -- 172.21.15.164:0/3498112491 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7effbc02a070 con 0x7effcc1019a0 2024-04-25T13:05:45.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.737+0000 7effbb7fe700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7effb40a3bb0 0x7effb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:05:45.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.737+0000 7effbb7fe700 1 -- 172.21.15.164:0/3498112491 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7effbc0cf850 con 0x7effcc1019a0 2024-04-25T13:05:45.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.737+0000 7effc9d9b700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7effb40a3bb0 0x7effb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:05:45.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.738+0000 7effc9d9b700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7effb40a3bb0 0x7effb40a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7effc0006fd0 tx=0x7effc0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:05:45.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.741+0000 7effbb7fe700 1 -- 172.21.15.164:0/3498112491 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effbc013070 con 0x7effcc1019a0 2024-04-25T13:05:45.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:45 smithi072 ceph-mon[29199]: osd.0 [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] boot 2024-04-25T13:05:45.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:45 smithi072 ceph-mon[29199]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:05:45.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:05:45.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:45 smithi072 ceph-mon[29199]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:05:45.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:05:45.947+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7effcc06b880 con 0x7effb40a3bb0 2024-04-25T13:05:46.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:46 smithi193 ceph-mon[32743]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:05:46.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:46 smithi193 ceph-mon[32743]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:05:46.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:05:46.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:05:46.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:46.755 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:46 smithi164 ceph-mon[32418]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:05:46.755 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:46 smithi164 ceph-mon[32418]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:05:46.755 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:05:46.755 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:05:46.755 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:46.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:46 smithi072 ceph-mon[29199]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:05:46.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:46 smithi072 ceph-mon[29199]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:05:46.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:05:46.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:05:46.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:47.762 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:47 smithi164 ceph-mon[32418]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:47.772 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:47 smithi072 ceph-mon[29199]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:47.901 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:47 smithi193 ceph-mon[32743]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:48.938 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:48 smithi164 ceph-mon[32418]: from='client.? 172.21.15.164:0/2587533473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2389db1-f136-4e7a-a5c9-bc63d97b938c"}]: dispatch 2024-04-25T13:05:48.939 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:48 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2389db1-f136-4e7a-a5c9-bc63d97b938c"}]: dispatch 2024-04-25T13:05:48.939 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:48 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2389db1-f136-4e7a-a5c9-bc63d97b938c"}]': finished 2024-04-25T13:05:48.939 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:48 smithi164 ceph-mon[32418]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:05:48.939 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:05:48.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:48 smithi193 ceph-mon[32743]: from='client.? 172.21.15.164:0/2587533473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2389db1-f136-4e7a-a5c9-bc63d97b938c"}]: dispatch 2024-04-25T13:05:48.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:48 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2389db1-f136-4e7a-a5c9-bc63d97b938c"}]: dispatch 2024-04-25T13:05:48.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:48 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2389db1-f136-4e7a-a5c9-bc63d97b938c"}]': finished 2024-04-25T13:05:48.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:48 smithi193 ceph-mon[32743]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:05:48.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:05:49.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:48 smithi072 ceph-mon[29199]: from='client.? 172.21.15.164:0/2587533473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2389db1-f136-4e7a-a5c9-bc63d97b938c"}]: dispatch 2024-04-25T13:05:49.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:48 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2389db1-f136-4e7a-a5c9-bc63d97b938c"}]: dispatch 2024-04-25T13:05:49.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:48 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2389db1-f136-4e7a-a5c9-bc63d97b938c"}]': finished 2024-04-25T13:05:49.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:48 smithi072 ceph-mon[29199]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:05:49.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:05:49.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:49 smithi193 ceph-mon[32743]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:49.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:49 smithi193 ceph-mon[32743]: from='client.? 172.21.15.164:0/4230261390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:05:50.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:49 smithi072 ceph-mon[29199]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:50.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:49 smithi072 ceph-mon[29199]: from='client.? 172.21.15.164:0/4230261390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:05:50.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:49 smithi164 ceph-mon[32418]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:50.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:49 smithi164 ceph-mon[32418]: from='client.? 172.21.15.164:0/4230261390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:05:52.474 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:52 smithi164 ceph-mon[32418]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:52.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:52 smithi193 ceph-mon[32743]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:52.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:52 smithi072 ceph-mon[29199]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:54.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:54 smithi164 ceph-mon[32418]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:54.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:54 smithi193 ceph-mon[32743]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:54.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:54 smithi072 ceph-mon[29199]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:56.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:56 smithi193 ceph-mon[32743]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:56.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:05:56.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:56 smithi072 ceph-mon[29199]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:05:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:56.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:56 smithi164 ceph-mon[32418]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:56.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:05:56.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:57.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:57 smithi193 ceph-mon[32743]: Deploying daemon osd.1 on smithi164 2024-04-25T13:05:57.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:57 smithi072 ceph-mon[29199]: Deploying daemon osd.1 on smithi164 2024-04-25T13:05:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:57 smithi164 ceph-mon[32418]: Deploying daemon osd.1 on smithi164 2024-04-25T13:05:58.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:05:58 smithi193 ceph-mon[32743]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:58.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:05:58 smithi072 ceph-mon[29199]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:05:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:05:58 smithi164 ceph-mon[32418]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:00.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:00 smithi193 ceph-mon[32743]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:00.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:00 smithi072 ceph-mon[29199]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:00.543 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:00 smithi164 ceph-mon[32418]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:02.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:02 smithi193 ceph-mon[32743]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:02.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:02 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:02.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:02 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:02.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:02 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:02.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:02 smithi072 ceph-mon[29199]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:02 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:02 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:02 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:02.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:02 smithi164 ceph-mon[32418]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:02 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:02 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:02 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:04.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:04 smithi164 ceph-mon[32418]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:04.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:04 smithi193 ceph-mon[32743]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:04.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:04 smithi072 ceph-mon[29199]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.811+0000 7effbb7fe700 1 -- 172.21.15.164:0/3498112491 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7effcc06b880 con 0x7effb40a3bb0 2024-04-25T13:06:04.811 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 1 on host 'smithi164' 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7effb40a3bb0 msgr2=0x7effb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7effb40a3bb0 0x7effb40a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7effc0006fd0 tx=0x7effc0008040 comp rx=0 tx=0).stop 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 msgr2=0x7effcc198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 0x7effcc198d40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effbc0098e0 tx=0x7effbc00fab0 comp rx=0 tx=0).stop 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 shutdown_connections 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7effb40a3bb0 0x7effb40a6070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effcc103540 0x7effcc1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc1019a0 0x7effcc198d40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 --2- 172.21.15.164:0/3498112491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effcc102ba0 0x7effcc199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 >> 172.21.15.164:0/3498112491 conn(0x7effcc0fcf40 msgr2=0x7effcc06b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:06:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 shutdown_connections 2024-04-25T13:06:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:04.814+0000 7effd0f5e700 1 -- 172.21.15.164:0/3498112491 wait complete. 2024-04-25T13:06:05.402 DEBUG:teuthology.orchestra.run.smithi164:osd.1> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T13:06:05.404 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2024-04-25T13:06:05.404 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:06:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:06:06.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:05 smithi164 ceph-mon[32418]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:05 smithi164 ceph-mon[32418]: from='osd.1 [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:06:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:05 smithi164 ceph-mon[32418]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:06:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:05 smithi193 ceph-mon[32743]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:06.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:05 smithi193 ceph-mon[32743]: from='osd.1 [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:06:06.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:05 smithi193 ceph-mon[32743]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:06:06.103 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.103 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:05 smithi072 ceph-mon[29199]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:06.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:05 smithi072 ceph-mon[29199]: from='osd.1 [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:06:06.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:05 smithi072 ceph-mon[29199]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:06:06.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:06.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:06 smithi193 ceph-mon[32743]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:06:06.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:06 smithi193 ceph-mon[32743]: from='osd.1 [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:06:06.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:06 smithi193 ceph-mon[32743]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:06:06.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:06 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:06.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:06 smithi193 ceph-mon[32743]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:06:07.096 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:06 smithi164 ceph-mon[32418]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:06:07.097 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:06 smithi164 ceph-mon[32418]: from='osd.1 [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:06:07.097 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:06 smithi164 ceph-mon[32418]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:06:07.097 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:06 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:07.097 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:06 smithi164 ceph-mon[32418]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:06:07.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:06 smithi072 ceph-mon[29199]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:06:07.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:06 smithi072 ceph-mon[29199]: from='osd.1 [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:06:07.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:06 smithi072 ceph-mon[29199]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:06:07.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:06 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:07.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:06 smithi072 ceph-mon[29199]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:06:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:06:07.531 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:06:07.531 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-04-25T13:06:07.531 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-04-25T13:06:07.531 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113305 s, 925 MB/s 2024-04-25T13:06:07.531 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-04-25T13:06:07.805 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:07 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1[37057]: 2024-04-25T13:06:07.715+0000 7f3a2391f700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T13:06:08.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:07 smithi193 ceph-mon[32743]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:06:08.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:07 smithi193 ceph-mon[32743]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:06:08.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:08.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:08.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:07 smithi193 ceph-mon[32743]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:08.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:07 smithi193 ceph-mon[32743]: from='osd.1 ' entity='osd.1' 2024-04-25T13:06:08.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:08.157 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-04-25T13:06:08.190 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:07 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1[37057]: 2024-04-25T13:06:07.878+0000 7f3a1b70a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:06:08.190 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:07 smithi164 ceph-mon[32418]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:06:08.190 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:07 smithi164 ceph-mon[32418]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:06:08.190 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:08.191 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:08.191 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:07 smithi164 ceph-mon[32418]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:08.191 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:07 smithi164 ceph-mon[32418]: from='osd.1 ' entity='osd.1' 2024-04-25T13:06:08.191 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:07 smithi072 ceph-mon[29199]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:06:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:07 smithi072 ceph-mon[29199]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:06:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:07 smithi072 ceph-mon[29199]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:07 smithi072 ceph-mon[29199]: from='osd.1 ' entity='osd.1' 2024-04-25T13:06:08.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:08.400 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:06:08.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:06:08.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:06:08.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:08.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:08.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:08.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:08.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:09.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:06:09.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:06:09.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:09.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:09.928 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.926+0000 7f35b7c6b700 1 -- 172.21.15.193:0/1505261978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 msgr2=0x7f35b010ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:09.929 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.926+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/1505261978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 0x7f35b010ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f35a800b0a0 tx=0x7f35a800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:06:09.929 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 -- 172.21.15.193:0/1505261978 shutdown_connections 2024-04-25T13:06:09.929 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/1505261978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 0x7f35b010ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:09.929 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/1505261978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35b01035c0 0x7f35b01059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:09.929 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/1505261978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b0105ef0 0x7f35b01082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:09.929 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 -- 172.21.15.193:0/1505261978 >> 172.21.15.193:0/1505261978 conn(0x7f35b00fcf70 msgr2=0x7f35b00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:06:09.929 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 -- 172.21.15.193:0/1505261978 shutdown_connections 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 -- 172.21.15.193:0/1505261978 wait complete. 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 Processor -- start 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 -- start start 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b01035c0 0x7f35b0198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35b0105ef0 0x7f35b0199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.927+0000 7f35b7c6b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 0x7f35b01a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b7c6b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f35b010e5e0 con 0x7f35b0105ef0 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b7c6b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35b010e460 con 0x7f35b01035c0 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b7c6b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35b010e760 con 0x7f35b0108820 2024-04-25T13:06:09.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b6208700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 0x7f35b01a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b6208700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 0x7f35b01a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:56852/0 (socket says 172.21.15.193:56852) 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b6208700 1 -- 172.21.15.193:0/2783457732 learned_addr learned my addr 172.21.15.193:0/2783457732 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b5206700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35b0105ef0 0x7f35b0199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b5a07700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b01035c0 0x7f35b0198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b6208700 1 -- 172.21.15.193:0/2783457732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b01035c0 msgr2=0x7f35b0198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b6208700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b01035c0 0x7f35b0198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b6208700 1 -- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35b0105ef0 msgr2=0x7f35b0199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b6208700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35b0105ef0 0x7f35b0199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b6208700 1 -- 172.21.15.193:0/2783457732 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b01a0890 con 0x7f35b0108820 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b6208700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 0x7f35b01a01c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009fd0 tx=0x7f35a8007770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35a6ffd700 1 -- 172.21.15.193:0/2783457732 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8021030 con 0x7f35b0108820 2024-04-25T13:06:09.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35a6ffd700 1 -- 172.21.15.193:0/2783457732 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a8007d20 con 0x7f35b0108820 2024-04-25T13:06:09.932 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b01a0b20 con 0x7f35b0108820 2024-04-25T13:06:09.932 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35a6ffd700 1 -- 172.21.15.193:0/2783457732 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a801d470 con 0x7f35b0108820 2024-04-25T13:06:09.932 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.928+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b01a0fd0 con 0x7f35b0108820 2024-04-25T13:06:09.932 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.929+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b0069730 con 0x7f35b0108820 2024-04-25T13:06:09.933 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.930+0000 7f35a6ffd700 1 -- 172.21.15.193:0/2783457732 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f35a8024070 con 0x7f35b0108820 2024-04-25T13:06:09.933 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.931+0000 7f35a6ffd700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f359c0a3c30 0x7f359c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:09.933 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.931+0000 7f35a6ffd700 1 -- 172.21.15.193:0/2783457732 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f35a80d0280 con 0x7f35b0108820 2024-04-25T13:06:09.933 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.931+0000 7f35b5a07700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f359c0a3c30 0x7f359c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:09.933 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.932+0000 7f35b5a07700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f359c0a3c30 0x7f359c0a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35b019a120 tx=0x7f35ac005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:06:09.933 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:09.935+0000 7f35a6ffd700 1 -- 172.21.15.193:0/2783457732 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35a809e540 con 0x7f35b0108820 2024-04-25T13:06:10.142 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:10.143+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f35b0062e80 con 0x7f359c0a3c30 2024-04-25T13:06:10.143 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi164 2024-04-25T13:06:10.143 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi164 to 17088M 2024-04-25T13:06:10.143 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: osd.1 [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] boot 2024-04-25T13:06:10.143 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:06:10.143 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:10.143 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:10.143 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:10.143 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:10.143 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:10.144 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:09 smithi193 ceph-mon[32743]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:10.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi164 2024-04-25T13:06:10.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi164 to 17088M 2024-04-25T13:06:10.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: osd.1 [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] boot 2024-04-25T13:06:10.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:06:10.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:10.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:10.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:10.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:10.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:10.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:09 smithi072 ceph-mon[29199]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:10.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi164 2024-04-25T13:06:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi164 to 17088M 2024-04-25T13:06:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: osd.1 [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] boot 2024-04-25T13:06:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:06:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:06:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:09 smithi164 ceph-mon[32418]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:06:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:11 smithi072 ceph-mon[29199]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:06:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:11 smithi072 ceph-mon[29199]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:06:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:06:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:06:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:11.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:11 smithi164 ceph-mon[32418]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:06:11.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:11 smithi164 ceph-mon[32418]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:06:11.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:06:11.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:06:11.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:11.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:11 smithi193 ceph-mon[32743]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:06:11.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:11 smithi193 ceph-mon[32743]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:06:11.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:06:11.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:06:11.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:12 smithi193 ceph-mon[32743]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:12.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:12 smithi072 ceph-mon[29199]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:12.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:12 smithi164 ceph-mon[32418]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:13 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/724463478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "778a3d43-534a-4aca-b391-07b8b9e045a4"}]: dispatch 2024-04-25T13:06:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:13 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "778a3d43-534a-4aca-b391-07b8b9e045a4"}]: dispatch 2024-04-25T13:06:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:13 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "778a3d43-534a-4aca-b391-07b8b9e045a4"}]': finished 2024-04-25T13:06:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:13 smithi072 ceph-mon[29199]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:06:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:13 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:13.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:13 smithi164 ceph-mon[32418]: from='client.? 172.21.15.193:0/724463478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "778a3d43-534a-4aca-b391-07b8b9e045a4"}]: dispatch 2024-04-25T13:06:13.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:13 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "778a3d43-534a-4aca-b391-07b8b9e045a4"}]: dispatch 2024-04-25T13:06:13.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:13 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "778a3d43-534a-4aca-b391-07b8b9e045a4"}]': finished 2024-04-25T13:06:13.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:13 smithi164 ceph-mon[32418]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:06:13.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:13 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:13.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:13 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/724463478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "778a3d43-534a-4aca-b391-07b8b9e045a4"}]: dispatch 2024-04-25T13:06:13.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:13 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "778a3d43-534a-4aca-b391-07b8b9e045a4"}]: dispatch 2024-04-25T13:06:13.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:13 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "778a3d43-534a-4aca-b391-07b8b9e045a4"}]': finished 2024-04-25T13:06:13.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:13 smithi193 ceph-mon[32743]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:06:13.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:13 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:14.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:14 smithi072 ceph-mon[29199]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:14.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:14 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/2370156208' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:06:14.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:14 smithi164 ceph-mon[32418]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:14.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:14 smithi164 ceph-mon[32418]: from='client.? 172.21.15.193:0/2370156208' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:06:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:14 smithi193 ceph-mon[32743]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:14 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/2370156208' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:06:16.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:16 smithi193 ceph-mon[32743]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:16 smithi072 ceph-mon[29199]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:16.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:16 smithi164 ceph-mon[32418]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:18.183 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:18 smithi193 ceph-mon[32743]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:18.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:18 smithi072 ceph-mon[29199]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:18.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:18 smithi164 ceph-mon[32418]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:20.435 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:20 smithi193 ceph-mon[32743]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:20.435 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:20 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:06:20.435 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:20 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:20.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:20 smithi072 ceph-mon[29199]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:20.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:06:20.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:20.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:20 smithi164 ceph-mon[32418]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:20.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:20 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:06:20.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:20 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:21.401 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:21 smithi193 ceph-mon[32743]: Deploying daemon osd.2 on smithi193 2024-04-25T13:06:21.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:21 smithi072 ceph-mon[29199]: Deploying daemon osd.2 on smithi193 2024-04-25T13:06:21.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:21 smithi164 ceph-mon[32418]: Deploying daemon osd.2 on smithi193 2024-04-25T13:06:22.397 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:22 smithi193 ceph-mon[32743]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:22.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:22 smithi072 ceph-mon[29199]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:22.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:22 smithi164 ceph-mon[32418]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:24.405 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:24 smithi193 ceph-mon[32743]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:24.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:24 smithi072 ceph-mon[29199]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:24.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:24 smithi164 ceph-mon[32418]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:26.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:25 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:26.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:25 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:26.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:25 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:26.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:25 smithi193 ceph-mon[32743]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:26.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:25 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:26.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:25 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:26.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:25 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:26.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:25 smithi072 ceph-mon[29199]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:26.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:25 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:26.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:25 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:26.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:25 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:26.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:25 smithi164 ceph-mon[32418]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:28 smithi193 ceph-mon[32743]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:28.254 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.255+0000 7f35a6ffd700 1 -- 172.21.15.193:0/2783457732 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f35b0062e80 con 0x7f359c0a3c30 2024-04-25T13:06:28.254 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2024-04-25T13:06:28.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f359c0a3c30 msgr2=0x7f359c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:28.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f359c0a3c30 0x7f359c0a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35b019a120 tx=0x7f35ac005d70 comp rx=0 tx=0).stop 2024-04-25T13:06:28.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 msgr2=0x7f35b01a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:28.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 0x7f35b01a01c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009fd0 tx=0x7f35a8007770 comp rx=0 tx=0).stop 2024-04-25T13:06:28.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 shutdown_connections 2024-04-25T13:06:28.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b0108820 0x7f35b01a01c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:28.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b01035c0 0x7f35b0198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:28.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f359c0a3c30 0x7f359c0a60f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:28.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 --2- 172.21.15.193:0/2783457732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35b0105ef0 0x7f35b0199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:28.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 >> 172.21.15.193:0/2783457732 conn(0x7f35b00fcf70 msgr2=0x7f35b0103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:06:28.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 shutdown_connections 2024-04-25T13:06:28.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:06:28.257+0000 7f35b7c6b700 1 -- 172.21.15.193:0/2783457732 wait complete. 2024-04-25T13:06:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:28 smithi072 ceph-mon[29199]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:28.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:28 smithi164 ceph-mon[32418]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:28.842 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T13:06:28.844 INFO:tasks.cephadm:Deploying osd.3 on smithi072 with /dev/vg_nvme/lv_3... 2024-04-25T13:06:28.844 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:06:28.980 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2[37928]: 2024-04-25T13:06:28.877+0000 7f84b1d627c0 -1 osd.2 0 log_to_monitors true 2024-04-25T13:06:29.086 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:06:29.275 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:29 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.275 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:29 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.275 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:29 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.276 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:29 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.276 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:29 smithi193 ceph-mon[32743]: from='osd.2 [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:06:29.276 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:29 smithi193 ceph-mon[32743]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:06:29.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:29 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:29 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:29 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:29 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:29 smithi072 ceph-mon[29199]: from='osd.2 [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:06:29.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:29 smithi072 ceph-mon[29199]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:06:29.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:29 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:29 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:29 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:29 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:29 smithi164 ceph-mon[32418]: from='osd.2 [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:06:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:29 smithi164 ceph-mon[32418]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:06:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:30 smithi072 ceph-mon[29199]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:30 smithi072 ceph-mon[29199]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:06:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:30 smithi072 ceph-mon[29199]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:06:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:30 smithi072 ceph-mon[29199]: from='osd.2 [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:06:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:30 smithi072 ceph-mon[29199]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:06:30.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:30 smithi193 ceph-mon[32743]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:30.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:30 smithi193 ceph-mon[32743]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:06:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:30 smithi193 ceph-mon[32743]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:06:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:30 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:30 smithi193 ceph-mon[32743]: from='osd.2 [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:06:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:30 smithi193 ceph-mon[32743]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:06:30.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:30 smithi164 ceph-mon[32418]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:30.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:30 smithi164 ceph-mon[32418]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:06:30.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:30 smithi164 ceph-mon[32418]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:06:30.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:30 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:30.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:30 smithi164 ceph-mon[32418]: from='osd.2 [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:06:30.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:30 smithi164 ceph-mon[32418]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:06:31.127 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:06:31.128 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:06:31.128 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-04-25T13:06:31.128 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-04-25T13:06:31.128 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114335 s, 917 MB/s 2024-04-25T13:06:31.128 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-04-25T13:06:31.335 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:31 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2[37928]: 2024-04-25T13:06:31.261+0000 7f84a73c7700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T13:06:31.644 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:31 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2[37928]: 2024-04-25T13:06:31.391+0000 7f849f1b2700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:06:31.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:31 smithi193 ceph-mon[32743]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:06:31.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:31 smithi193 ceph-mon[32743]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:06:31.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:31.645 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:31.645 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:31 smithi193 ceph-mon[32743]: from='osd.2 ' entity='osd.2' 2024-04-25T13:06:31.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:31 smithi072 ceph-mon[29199]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:06:31.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:31 smithi072 ceph-mon[29199]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:06:31.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:31.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:31.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:31 smithi072 ceph-mon[29199]: from='osd.2 ' entity='osd.2' 2024-04-25T13:06:31.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:31 smithi164 ceph-mon[32418]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:06:31.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:31 smithi164 ceph-mon[32418]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:06:31.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:31.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:31.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:31 smithi164 ceph-mon[32418]: from='osd.2 ' entity='osd.2' 2024-04-25T13:06:31.840 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_3 2024-04-25T13:06:32.089 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:06:32.519 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:06:32.519 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:06:32.519 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:32.519 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:32.519 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.519 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.519 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:32.519 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.519 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:32.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:32.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:32.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:32.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:32.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:32.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:06:32.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:06:32.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:32.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:06:32.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:06:32.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:32.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:32.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:32.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:32.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:32.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:32.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:32.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:32.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:32.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:33.438 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:33 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi193 2024-04-25T13:06:33.438 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:33 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi193 to 17088M 2024-04-25T13:06:33.439 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:33 smithi072 ceph-mon[29199]: osd.2 [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] boot 2024-04-25T13:06:33.439 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:33 smithi072 ceph-mon[29199]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:06:33.439 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:33 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:33.439 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:33 smithi072 ceph-mon[29199]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:33.439 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:33 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:06:33.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.644+0000 7ffb244f1700 1 -- 172.21.15.72:0/1476358636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c103560 msgr2=0x7ffb1c109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:33.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.644+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1476358636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c103560 0x7ffb1c109e10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800b0d0 tx=0x7ffb1800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:06:33.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.644+0000 7ffb244f1700 1 -- 172.21.15.72:0/1476358636 shutdown_connections 2024-04-25T13:06:33.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.644+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1476358636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c103560 0x7ffb1c109e10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:33.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.644+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1476358636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb1c1019c0 0x7ffb1c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.644+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1476358636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb1c102bc0 0x7ffb1c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.644+0000 7ffb244f1700 1 -- 172.21.15.72:0/1476358636 >> 172.21.15.72:0/1476358636 conn(0x7ffb1c0fcfa0 msgr2=0x7ffb1c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.644+0000 7ffb244f1700 1 -- 172.21.15.72:0/1476358636 shutdown_connections 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.644+0000 7ffb244f1700 1 -- 172.21.15.72:0/1476358636 wait complete. 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb244f1700 1 Processor -- start 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb244f1700 1 -- start start 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb244f1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c1019c0 0x7ffb1c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb244f1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb1c102bc0 0x7ffb1c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb244f1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb1c103560 0x7ffb1c1a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb244f1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c10d520 con 0x7ffb1c1019c0 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb244f1700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c10d3a0 con 0x7ffb1c102bc0 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb244f1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c10d6a0 con 0x7ffb1c103560 2024-04-25T13:06:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb2228d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c1019c0 0x7ffb1c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb2228d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c1019c0 0x7ffb1c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46396/0 (socket says 172.21.15.72:46396) 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb2228d700 1 -- 172.21.15.72:0/1113558841 learned_addr learned my addr 172.21.15.72:0/1113558841 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb22a8e700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb1c103560 0x7ffb1c1a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.645+0000 7ffb21a8c700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb1c102bc0 0x7ffb1c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb2228d700 1 -- 172.21.15.72:0/1113558841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb1c103560 msgr2=0x7ffb1c1a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb2228d700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb1c103560 0x7ffb1c1a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb2228d700 1 -- 172.21.15.72:0/1113558841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb1c102bc0 msgr2=0x7ffb1c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb2228d700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb1c102bc0 0x7ffb1c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb2228d700 1 -- 172.21.15.72:0/1113558841 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c1a0890 con 0x7ffb1c1019c0 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb21a8c700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb1c102bc0 0x7ffb1c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb2228d700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c1019c0 0x7ffb1c198c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400b500 tx=0x7ffb1400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb137fe700 1 -- 172.21.15.72:0/1113558841 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14015020 con 0x7ffb1c1019c0 2024-04-25T13:06:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c1a0b80 con 0x7ffb1c1019c0 2024-04-25T13:06:33.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb137fe700 1 -- 172.21.15.72:0/1113558841 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb14011ce0 con 0x7ffb1c1019c0 2024-04-25T13:06:33.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb137fe700 1 -- 172.21.15.72:0/1113558841 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14010d20 con 0x7ffb1c1019c0 2024-04-25T13:06:33.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.646+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c1a10c0 con 0x7ffb1c1019c0 2024-04-25T13:06:33.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.647+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c069730 con 0x7ffb1c1019c0 2024-04-25T13:06:33.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.648+0000 7ffb137fe700 1 -- 172.21.15.72:0/1113558841 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ffb14018020 con 0x7ffb1c1019c0 2024-04-25T13:06:33.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.649+0000 7ffb137fe700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb080a3bb0 0x7ffb080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:33.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.649+0000 7ffb21a8c700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb080a3bb0 0x7ffb080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:33.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.649+0000 7ffb137fe700 1 -- 172.21.15.72:0/1113558841 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7ffb140cebe0 con 0x7ffb1c1019c0 2024-04-25T13:06:33.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.649+0000 7ffb21a8c700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb080a3bb0 0x7ffb080a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c005950 tx=0x7ffb0c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:06:33.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.652+0000 7ffb137fe700 1 -- 172.21.15.72:0/1113558841 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb1401f020 con 0x7ffb1c1019c0 2024-04-25T13:06:33.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:33 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi193 2024-04-25T13:06:33.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:33 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi193 to 17088M 2024-04-25T13:06:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:33 smithi193 ceph-mon[32743]: osd.2 [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] boot 2024-04-25T13:06:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:33 smithi193 ceph-mon[32743]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:06:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:33 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:33 smithi193 ceph-mon[32743]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:33 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:06:33.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:33 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi193 2024-04-25T13:06:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:33 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi193 to 17088M 2024-04-25T13:06:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:33 smithi164 ceph-mon[32418]: osd.2 [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] boot 2024-04-25T13:06:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:33 smithi164 ceph-mon[32418]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:06:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:33 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:06:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:33 smithi164 ceph-mon[32418]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:06:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:33 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:06:33.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:33.863+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ffb1c0ff0e0 con 0x7ffb080a3bb0 2024-04-25T13:06:34.609 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:06:34.610 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 ceph-mon[29199]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:06:34.610 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:06:34.610 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 ceph-mon[29199]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:06:34.610 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:06:34.610 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:06:34.610 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:34.610 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49090]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:06:34.610 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49090]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:34.610 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49090]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:34.611 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49090]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:34.611 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49093]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:06:34.611 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49093]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:34.611 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49093]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:34.611 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49093]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:34.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:06:34.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 ceph-mon[32743]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:06:34.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:06:34.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 ceph-mon[32743]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:06:34.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:06:34.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:06:34.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:34.750 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:06:34.750 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 ceph-mon[32418]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:06:34.750 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:06:34.750 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 ceph-mon[32418]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:06:34.750 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:06:34.750 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:06:34.750 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:34.751 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40885]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:06:34.751 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40885]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:34.751 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40885]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:34.751 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40885]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:34.751 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40888]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:06:34.751 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40888]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:34.751 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40888]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:34.751 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40888]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:34.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41543]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:06:34.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:34.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41543]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:34.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41543]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:34.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41546]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:06:34.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41546]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:34.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41546]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:34.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41546]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:34.981 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41537]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:06:34.981 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41537]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:34.981 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41537]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:34.982 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41537]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:34.982 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41540]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:06:34.982 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41540]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:34.982 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41540]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:34.982 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:06:34 smithi193 sudo[41540]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:35.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:06:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49100]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49100]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49112]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:06:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49112]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49112]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:34 smithi072 sudo[49112]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:35.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40891]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:06:35.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40891]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:35.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40891]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:35.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40891]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:35.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40894]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:06:35.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40894]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:06:35.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40894]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:06:35.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:34 smithi164 sudo[40894]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:06:35.622 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:06:35.695 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:06:35.696 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:35.696 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:35.696 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:35.696 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:06:35.696 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:06:35.696 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:06:35.696 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:06:35.696 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:06:35.697 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:35 smithi072 ceph-mon[29199]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:35.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:35.674+0000 7ffb137fe700 1 -- 172.21.15.72:0/1113558841 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffb14010380 con 0x7ffb1c1019c0 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:06:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:06:35.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:35 smithi193 ceph-mon[32743]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:06:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:06:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:06:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:06:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:06:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:06:35.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:06:35.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:35 smithi164 ceph-mon[32418]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:36.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:36 smithi193 ceph-mon[32743]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:06:36.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:36 smithi193 ceph-mon[32743]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:06:36.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:36 smithi072 ceph-mon[29199]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:06:36.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:36 smithi072 ceph-mon[29199]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:06:36.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:36 smithi164 ceph-mon[32418]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:06:36.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:36 smithi164 ceph-mon[32418]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:06:37.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:37 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2428643664' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2134ce99-2a05-49fd-8653-81180a5c199a"}]: dispatch 2024-04-25T13:06:37.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:37 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2428643664' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2134ce99-2a05-49fd-8653-81180a5c199a"}]': finished 2024-04-25T13:06:37.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:37 smithi193 ceph-mon[32743]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T13:06:37.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:37 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:37.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:37 smithi193 ceph-mon[32743]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:37.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:37 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1056370863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:06:37.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:37 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2428643664' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2134ce99-2a05-49fd-8653-81180a5c199a"}]: dispatch 2024-04-25T13:06:37.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:37 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2428643664' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2134ce99-2a05-49fd-8653-81180a5c199a"}]': finished 2024-04-25T13:06:37.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:37 smithi072 ceph-mon[29199]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T13:06:37.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:37 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:37.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:37 smithi072 ceph-mon[29199]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:37.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:37 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1056370863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:06:37.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:37 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2428643664' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2134ce99-2a05-49fd-8653-81180a5c199a"}]: dispatch 2024-04-25T13:06:37.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:37 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2428643664' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2134ce99-2a05-49fd-8653-81180a5c199a"}]': finished 2024-04-25T13:06:37.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:37 smithi164 ceph-mon[32418]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T13:06:37.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:37 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:37.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:37 smithi164 ceph-mon[32418]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:37.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:37 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1056370863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:06:40.470 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:40 smithi072 ceph-mon[29199]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:40.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:40 smithi193 ceph-mon[32743]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:40.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:40 smithi164 ceph-mon[32418]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:42.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:42 smithi072 ceph-mon[29199]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:42.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:42 smithi193 ceph-mon[32743]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:42.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:42 smithi164 ceph-mon[32418]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:44.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:44 smithi193 ceph-mon[32743]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:44.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:06:44.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:44.522 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:44 smithi072 ceph-mon[29199]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:44.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:06:44.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:44.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:44 smithi164 ceph-mon[32418]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:44.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:06:44.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:45.240 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:45 smithi072 ceph-mon[29199]: Deploying daemon osd.3 on smithi072 2024-04-25T13:06:45.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:45 smithi193 ceph-mon[32743]: Deploying daemon osd.3 on smithi072 2024-04-25T13:06:45.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:45 smithi164 ceph-mon[32418]: Deploying daemon osd.3 on smithi072 2024-04-25T13:06:46.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:46 smithi193 ceph-mon[32743]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:46.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:46 smithi072 ceph-mon[29199]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:46.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:46 smithi164 ceph-mon[32418]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:48.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:48 smithi193 ceph-mon[32743]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:48.514 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:48 smithi072 ceph-mon[29199]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:48.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:48 smithi164 ceph-mon[32418]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:50.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:50 smithi072 ceph-mon[29199]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:50.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:50.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:50.339 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:50.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:50 smithi193 ceph-mon[32743]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:50.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:50.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:50.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:50.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:50 smithi164 ceph-mon[32418]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:50.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:50.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:50.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:52.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:52 smithi193 ceph-mon[32743]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:52.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:52 smithi072 ceph-mon[29199]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:52.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:52 smithi164 ceph-mon[32418]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:53.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.022+0000 7ffb137fe700 1 -- 172.21.15.72:0/1113558841 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffb1c0ff0e0 con 0x7ffb080a3bb0 2024-04-25T13:06:53.021 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 3 on host 'smithi072' 2024-04-25T13:06:53.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb080a3bb0 msgr2=0x7ffb080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb080a3bb0 0x7ffb080a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c005950 tx=0x7ffb0c0058e0 comp rx=0 tx=0).stop 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c1019c0 msgr2=0x7ffb1c198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c1019c0 0x7ffb1c198c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400b500 tx=0x7ffb1400b810 comp rx=0 tx=0).stop 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 shutdown_connections 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb1c103560 0x7ffb1c1a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb080a3bb0 0x7ffb080a6070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb1c1019c0 0x7ffb1c198c40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 --2- 172.21.15.72:0/1113558841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb1c102bc0 0x7ffb1c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 >> 172.21.15.72:0/1113558841 conn(0x7ffb1c0fcfa0 msgr2=0x7ffb1c0fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 shutdown_connections 2024-04-25T13:06:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:06:53.024+0000 7ffb244f1700 1 -- 172.21.15.72:0/1113558841 wait complete. 2024-04-25T13:06:53.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:53.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:53.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:53.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:53.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:53.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:53.687 DEBUG:teuthology.orchestra.run.smithi072:osd.3> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T13:06:53.689 INFO:tasks.cephadm:Deploying osd.4 on smithi164 with /dev/vg_nvme/lv_3... 2024-04-25T13:06:53.689 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:06:53.935 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:06:54.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:54 smithi072 ceph-mon[29199]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:54.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:54 smithi072 ceph-mon[29199]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:06:54.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:54 smithi164 ceph-mon[32418]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:54.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:54 smithi164 ceph-mon[32418]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:06:54.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:54 smithi193 ceph-mon[32743]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:54.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:54 smithi193 ceph-mon[32743]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:06:55.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:55 smithi164 ceph-mon[32418]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:06:55.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:55 smithi164 ceph-mon[32418]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:06:55.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:55 smithi164 ceph-mon[32418]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:06:55.257 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:55 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:55.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:55 smithi193 ceph-mon[32743]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:06:55.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:55 smithi193 ceph-mon[32743]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:06:55.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:55 smithi193 ceph-mon[32743]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:06:55.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:55 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:55.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:55 smithi072 ceph-mon[29199]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:06:55.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:55 smithi072 ceph-mon[29199]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:06:55.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:55 smithi072 ceph-mon[29199]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:06:55.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:55 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:55.524 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:06:55 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3[51236]: 2024-04-25T13:06:55.173+0000 7f9699be5700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T13:06:55.524 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:06:55 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3[51236]: 2024-04-25T13:06:55.178+0000 7f96959d8700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:06:55.904 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:06:55.904 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:06:55.904 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-25T13:06:55.904 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-25T13:06:55.904 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138897 s, 755 MB/s 2024-04-25T13:06:55.904 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-25T13:06:56.480 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:56 smithi164 ceph-mon[32418]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:56.480 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:56 smithi164 ceph-mon[32418]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:06:56.480 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:56 smithi164 ceph-mon[32418]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:06:56.480 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:56.480 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:56.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:56 smithi193 ceph-mon[32743]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:56.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:56 smithi193 ceph-mon[32743]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:06:56.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:56 smithi193 ceph-mon[32743]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:06:56.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:56.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:56.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2024-04-25T13:06:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:56 smithi072 ceph-mon[29199]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:56 smithi072 ceph-mon[29199]: from='osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:06:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:56 smithi072 ceph-mon[29199]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:06:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:56.757 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] boot 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:57.453 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:57.454 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.454 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:57.454 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:57.454 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:06:57.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:06:57.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] boot 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:57.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:06:57.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:06:57.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:57.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: osd.3 [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] boot 2024-04-25T13:06:57.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T13:06:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:06:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:06:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:06:58.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.222+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/3656386435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 msgr2=0x7f8ef4105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:58.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.222+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/3656386435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 0x7f8ef4105940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009990 tx=0x7f8ee4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:06:58.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.223+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/3656386435 shutdown_connections 2024-04-25T13:06:58.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.223+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/3656386435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ef41087b0 0x7f8ef410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:58.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.223+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/3656386435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 0x7f8ef4105940 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:58.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.223+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/3656386435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ef4105e80 0x7f8ef4108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:58.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.223+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/3656386435 >> 172.21.15.164:0/3656386435 conn(0x7f8ef40fcfa0 msgr2=0x7f8ef40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:06:58.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.223+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/3656386435 shutdown_connections 2024-04-25T13:06:58.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.223+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/3656386435 wait complete. 2024-04-25T13:06:58.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.223+0000 7f8ef8fb2700 1 Processor -- start 2024-04-25T13:06:58.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef8fb2700 1 -- start start 2024-04-25T13:06:58.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef8fb2700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 0x7f8ef4102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:58.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef8fb2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ef4105e80 0x7f8ef4101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:58.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef8fb2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ef41087b0 0x7f8ef4101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:58.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef8fb2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef410e560 con 0x7f8ef4105e80 2024-04-25T13:06:58.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef8fb2700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef410e3e0 con 0x7f8ef4103550 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef8fb2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef410e6e0 con 0x7f8ef41087b0 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 0x7f8ef4102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 0x7f8ef4102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60774/0 (socket says 172.21.15.164:60774) 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef259c700 1 -- 172.21.15.164:0/777398206 learned_addr learned my addr 172.21.15.164:0/777398206 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef2d9d700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ef41087b0 0x7f8ef4101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef1d9b700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ef4105e80 0x7f8ef4101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef259c700 1 -- 172.21.15.164:0/777398206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ef41087b0 msgr2=0x7f8ef4101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef259c700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ef41087b0 0x7f8ef4101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef259c700 1 -- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ef4105e80 msgr2=0x7f8ef4101500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef259c700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ef4105e80 0x7f8ef4101500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef259c700 1 -- 172.21.15.164:0/777398206 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef41022f0 con 0x7f8ef4103550 2024-04-25T13:06:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ef259c700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 0x7f8ef4102eb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4006010 tx=0x7f8ee40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:06:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.224+0000 7f8ee37fe700 1 -- 172.21.15.164:0/777398206 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4027070 con 0x7f8ef4103550 2024-04-25T13:06:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.225+0000 7f8ee37fe700 1 -- 172.21.15.164:0/777398206 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee400fbc0 con 0x7f8ef4103550 2024-04-25T13:06:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.225+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef41a2d30 con 0x7f8ef4103550 2024-04-25T13:06:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.225+0000 7f8ee37fe700 1 -- 172.21.15.164:0/777398206 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4017650 con 0x7f8ef4103550 2024-04-25T13:06:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.225+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef41a3180 con 0x7f8ef4103550 2024-04-25T13:06:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.225+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef4069730 con 0x7f8ef4103550 2024-04-25T13:06:58.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.227+0000 7f8ee37fe700 1 -- 172.21.15.164:0/777398206 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8ee4009d10 con 0x7f8ef4103550 2024-04-25T13:06:58.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.227+0000 7f8ee37fe700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8edc0a3c40 0x7f8edc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:06:58.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.227+0000 7f8ee37fe700 1 -- 172.21.15.164:0/777398206 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f8ee40d0290 con 0x7f8ef4103550 2024-04-25T13:06:58.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.228+0000 7f8ef1d9b700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8edc0a3c40 0x7f8edc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:06:58.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.228+0000 7f8ef1d9b700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8edc0a3c40 0x7f8edc0a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009bf0 tx=0x7f8ee8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:06:58.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.231+0000 7f8ee37fe700 1 -- 172.21.15.164:0/777398206 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ee4013070 con 0x7f8ef4103550 2024-04-25T13:06:58.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:58 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi072 2024-04-25T13:06:58.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:58 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi072 to 8544M 2024-04-25T13:06:58.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:58 smithi164 ceph-mon[32418]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:58.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:58 smithi164 ceph-mon[32418]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T13:06:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:06:58.439+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8ef4062e80 con 0x7f8edc0a3c40 2024-04-25T13:06:58.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:58 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi072 2024-04-25T13:06:58.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:58 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi072 to 8544M 2024-04-25T13:06:58.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:58 smithi193 ceph-mon[32743]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:58.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:58 smithi193 ceph-mon[32743]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T13:06:58.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:58 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi072 2024-04-25T13:06:58.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:58 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi072 to 8544M 2024-04-25T13:06:58.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:58 smithi072 ceph-mon[29199]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:06:58.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:58 smithi072 ceph-mon[29199]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T13:06:59.284 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:59 smithi164 ceph-mon[32418]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:06:59.284 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:06:59.284 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:06:59.284 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:06:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:59.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:59 smithi193 ceph-mon[32743]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:06:59.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:06:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:06:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:06:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:59 smithi072 ceph-mon[29199]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:06:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:06:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:06:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:06:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:00.460 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:00 smithi072 ceph-mon[29199]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:07:00.460 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:00 smithi072 ceph-mon[29199]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:00.460 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:00 smithi072 ceph-mon[29199]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:07:00.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:00 smithi193 ceph-mon[32743]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:07:00.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:00 smithi193 ceph-mon[32743]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:00.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:00 smithi193 ceph-mon[32743]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:07:00.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:00 smithi164 ceph-mon[32418]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:07:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:00 smithi164 ceph-mon[32418]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:00 smithi164 ceph-mon[32418]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:07:01.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:01 smithi193 ceph-mon[32743]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T13:07:01.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:01 smithi193 ceph-mon[32743]: from='client.? 172.21.15.164:0/337368078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d3bd39-e0cb-403c-8911-1a633198aa3d"}]: dispatch 2024-04-25T13:07:01.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:01 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d3bd39-e0cb-403c-8911-1a633198aa3d"}]: dispatch 2024-04-25T13:07:01.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:01 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79d3bd39-e0cb-403c-8911-1a633198aa3d"}]': finished 2024-04-25T13:07:01.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:01 smithi193 ceph-mon[32743]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T13:07:01.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:01.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:01 smithi072 ceph-mon[29199]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T13:07:01.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:01 smithi072 ceph-mon[29199]: from='client.? 172.21.15.164:0/337368078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d3bd39-e0cb-403c-8911-1a633198aa3d"}]: dispatch 2024-04-25T13:07:01.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:01 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d3bd39-e0cb-403c-8911-1a633198aa3d"}]: dispatch 2024-04-25T13:07:01.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:01 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79d3bd39-e0cb-403c-8911-1a633198aa3d"}]': finished 2024-04-25T13:07:01.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:01 smithi072 ceph-mon[29199]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T13:07:01.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:01.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:01 smithi164 ceph-mon[32418]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T13:07:01.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:01 smithi164 ceph-mon[32418]: from='client.? 172.21.15.164:0/337368078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d3bd39-e0cb-403c-8911-1a633198aa3d"}]: dispatch 2024-04-25T13:07:01.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:01 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d3bd39-e0cb-403c-8911-1a633198aa3d"}]: dispatch 2024-04-25T13:07:01.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:01 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79d3bd39-e0cb-403c-8911-1a633198aa3d"}]': finished 2024-04-25T13:07:01.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:01 smithi164 ceph-mon[32418]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T13:07:01.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:02.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:02 smithi193 ceph-mon[32743]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-04-25T13:07:02.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:02 smithi193 ceph-mon[32743]: from='client.? 172.21.15.164:0/3386975612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:07:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:02 smithi072 ceph-mon[29199]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-04-25T13:07:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:02 smithi072 ceph-mon[29199]: from='client.? 172.21.15.164:0/3386975612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:07:02.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:02 smithi164 ceph-mon[32418]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-04-25T13:07:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:02 smithi164 ceph-mon[32418]: from='client.? 172.21.15.164:0/3386975612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:07:04.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:04 smithi193 ceph-mon[32743]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T13:07:04.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:04 smithi072 ceph-mon[29199]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T13:07:04.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:04 smithi164 ceph-mon[32418]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T13:07:06.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:06 smithi193 ceph-mon[32743]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:07:06.509 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:06 smithi164 ceph-mon[32418]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:07:06.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:06 smithi072 ceph-mon[29199]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:07:08.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:08 smithi164 ceph-mon[32418]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T13:07:08.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:08 smithi193 ceph-mon[32743]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T13:07:08.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:08 smithi072 ceph-mon[29199]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T13:07:09.437 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:09 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:07:09.437 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:09 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:09.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:09 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:07:09.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:09 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:09.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:07:09.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:10.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:10 smithi164 ceph-mon[32418]: Deploying daemon osd.4 on smithi164 2024-04-25T13:07:10.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:10 smithi164 ceph-mon[32418]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:10.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:10 smithi193 ceph-mon[32743]: Deploying daemon osd.4 on smithi164 2024-04-25T13:07:10.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:10 smithi193 ceph-mon[32743]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:10.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:10 smithi072 ceph-mon[29199]: Deploying daemon osd.4 on smithi164 2024-04-25T13:07:10.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:10 smithi072 ceph-mon[29199]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:12.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:12 smithi164 ceph-mon[32418]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:12.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:12 smithi193 ceph-mon[32743]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:12.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:12 smithi072 ceph-mon[29199]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:14 smithi193 ceph-mon[32743]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:14.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:14 smithi072 ceph-mon[29199]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:14 smithi164 ceph-mon[32418]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:16.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:16 smithi164 ceph-mon[32418]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:16.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:16 smithi193 ceph-mon[32743]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:16 smithi072 ceph-mon[29199]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:17.151 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 4 on host 'smithi164' 2024-04-25T13:07:17.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.151+0000 7f8ee37fe700 1 -- 172.21.15.164:0/777398206 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8ef4062e80 con 0x7f8edc0a3c40 2024-04-25T13:07:17.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.154+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8edc0a3c40 msgr2=0x7f8edc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:17.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.154+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8edc0a3c40 0x7f8edc0a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009bf0 tx=0x7f8ee8009260 comp rx=0 tx=0).stop 2024-04-25T13:07:17.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.154+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 msgr2=0x7f8ef4102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:17.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.154+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 0x7f8ef4102eb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4006010 tx=0x7f8ee40047b0 comp rx=0 tx=0).stop 2024-04-25T13:07:17.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.155+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 shutdown_connections 2024-04-25T13:07:17.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.155+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ef41087b0 0x7f8ef4101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:17.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.155+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef4103550 0x7f8ef4102eb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:17.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.155+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8edc0a3c40 0x7f8edc0a6100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:17.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.155+0000 7f8ef8fb2700 1 --2- 172.21.15.164:0/777398206 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ef4105e80 0x7f8ef4101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:17.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.155+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 >> 172.21.15.164:0/777398206 conn(0x7f8ef40fcfa0 msgr2=0x7f8ef4104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:07:17.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.155+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 shutdown_connections 2024-04-25T13:07:17.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:07:17.155+0000 7f8ef8fb2700 1 -- 172.21.15.164:0/777398206 wait complete. 2024-04-25T13:07:17.833 DEBUG:teuthology.orchestra.run.smithi164:osd.4> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T13:07:17.835 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-04-25T13:07:17.835 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:07:18.104 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:07:18.359 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:18 smithi164 ceph-mon[32418]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:18.359 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:18 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.359 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:18 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.360 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:18 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.360 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:18 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.360 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:18 smithi164 ceph-mon[32418]: from='osd.4 [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:07:18.360 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:18 smithi164 ceph-mon[32418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:07:18.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:18 smithi193 ceph-mon[32743]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:18.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:18 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:18 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:18 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:18 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:18 smithi193 ceph-mon[32743]: from='osd.4 [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:07:18.470 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:18 smithi193 ceph-mon[32743]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:07:18.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:18 smithi072 ceph-mon[29199]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:18.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:18 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:18 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:18 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:18 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:18.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:18 smithi072 ceph-mon[29199]: from='osd.4 [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:07:18.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:18 smithi072 ceph-mon[29199]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:07:19.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:19 smithi193 ceph-mon[32743]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:07:19.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:19 smithi193 ceph-mon[32743]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T13:07:19.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:19 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:19.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:19 smithi193 ceph-mon[32743]: from='osd.4 [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:07:19.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:19 smithi193 ceph-mon[32743]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:07:19.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:19 smithi193 ceph-mon[32743]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:19 smithi164 ceph-mon[32418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:07:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:19 smithi164 ceph-mon[32418]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T13:07:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:19 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:19 smithi164 ceph-mon[32418]: from='osd.4 [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:07:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:19 smithi164 ceph-mon[32418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:07:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:19 smithi164 ceph-mon[32418]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:19.538 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:07:19 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4[43693]: 2024-04-25T13:07:19.477+0000 7f9c01651700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T13:07:19.538 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:07:19 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4[43693]: 2024-04-25T13:07:19.483+0000 7f9bfd444700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:07:19.608 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:19 smithi072 ceph-mon[29199]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:07:19.608 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:19 smithi072 ceph-mon[29199]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T13:07:19.608 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:19 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:19.608 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:19 smithi072 ceph-mon[29199]: from='osd.4 [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:07:19.608 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:19 smithi072 ceph-mon[29199]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:07:19.608 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:19 smithi072 ceph-mon[29199]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:07:19.932 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:07:19.932 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:07:19.932 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-04-25T13:07:19.932 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-04-25T13:07:19.932 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114175 s, 918 MB/s 2024-04-25T13:07:19.932 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-04-25T13:07:20.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:20 smithi193 ceph-mon[32743]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:07:20.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:20 smithi193 ceph-mon[32743]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T13:07:20.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:20 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:20.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:20 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:20.511 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-04-25T13:07:20.703 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:20 smithi164 ceph-mon[32418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:07:20.703 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:20 smithi164 ceph-mon[32418]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T13:07:20.703 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:20 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:20.703 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:20 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:20.753 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:07:20.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:20 smithi072 ceph-mon[29199]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:07:20.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:20 smithi072 ceph-mon[29199]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T13:07:20.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:20.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:07:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:07:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: osd.4 [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] boot 2024-04-25T13:07:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T13:07:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi164 2024-04-25T13:07:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi164 to 8544M 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:21.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:21 smithi193 ceph-mon[32743]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:21.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:07:21.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: osd.4 [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] boot 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi164 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi164 to 8544M 2024-04-25T13:07:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:21.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:21.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:21.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:21.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:21.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:21 smithi072 ceph-mon[29199]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:21.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:07:21.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:07:21.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: osd.4 [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] boot 2024-04-25T13:07:21.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T13:07:21.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:07:21.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi164 2024-04-25T13:07:21.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi164 to 8544M 2024-04-25T13:07:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:21.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:21.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:21.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:21.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:21 smithi164 ceph-mon[32418]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:22.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.211+0000 7f695dc13700 1 -- 172.21.15.193:0/3488591000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f695810a9a0 msgr2=0x7f695810ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:22.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.211+0000 7f695dc13700 1 --2- 172.21.15.193:0/3488591000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f695810a9a0 0x7f695810ce10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f695000b0a0 tx=0x7f695000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:07:22.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.211+0000 7f695dc13700 1 -- 172.21.15.193:0/3488591000 shutdown_connections 2024-04-25T13:07:22.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.211+0000 7f695dc13700 1 --2- 172.21.15.193:0/3488591000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f695810a9a0 0x7f695810ce10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:22.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.211+0000 7f695dc13700 1 --2- 172.21.15.193:0/3488591000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69580769f0 0x7f6958074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:22.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.211+0000 7f695dc13700 1 --2- 172.21.15.193:0/3488591000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6958075420 0x7f69580758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:22.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.211+0000 7f695dc13700 1 -- 172.21.15.193:0/3488591000 >> 172.21.15.193:0/3488591000 conn(0x7f69580fe040 msgr2=0x7f6958100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:07:22.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.211+0000 7f695dc13700 1 -- 172.21.15.193:0/3488591000 shutdown_connections 2024-04-25T13:07:22.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.211+0000 7f695dc13700 1 -- 172.21.15.193:0/3488591000 wait complete. 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f695dc13700 1 Processor -- start 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f695dc13700 1 -- start start 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f695dc13700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6958075420 0x7f695819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f695dc13700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69580769f0 0x7f695819b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f695dc13700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f695810a9a0 0x7f69581a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f695dc13700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f69581106f0 con 0x7f69580769f0 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f695dc13700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6958110570 con 0x7f695810a9a0 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f695dc13700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6958110870 con 0x7f6958075420 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f69577fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6958075420 0x7f695819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f69577fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6958075420 0x7f695819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:45382/0 (socket says 172.21.15.193:45382) 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.212+0000 7f69577fe700 1 -- 172.21.15.193:0/1439924779 learned_addr learned my addr 172.21.15.193:0/1439924779 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:07:22.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f6956ffd700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69580769f0 0x7f695819b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f6957fff700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f695810a9a0 0x7f69581a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f69577fe700 1 -- 172.21.15.193:0/1439924779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f695810a9a0 msgr2=0x7f69581a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f69577fe700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f695810a9a0 0x7f69581a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f69577fe700 1 -- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69580769f0 msgr2=0x7f695819b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f69577fe700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69580769f0 0x7f695819b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f69577fe700 1 -- 172.21.15.193:0/1439924779 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69581a2910 con 0x7f6958075420 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f69577fe700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6958075420 0x7f695819ac90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f694800c9b0 tx=0x7f694800ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f6954ff9700 1 -- 172.21.15.193:0/1439924779 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948007b00 con 0x7f6958075420 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69581a2c00 con 0x7f6958075420 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f6954ff9700 1 -- 172.21.15.193:0/1439924779 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6948007ca0 con 0x7f6958075420 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f6954ff9700 1 -- 172.21.15.193:0/1439924779 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948018660 con 0x7f6958075420 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.213+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69581a3190 con 0x7f6958075420 2024-04-25T13:07:22.215 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.214+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6958069730 con 0x7f6958075420 2024-04-25T13:07:22.216 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.215+0000 7f6954ff9700 1 -- 172.21.15.193:0/1439924779 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f694800f450 con 0x7f6958075420 2024-04-25T13:07:22.216 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.216+0000 7f6954ff9700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f69400a3c40 0x7f69400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:07:22.216 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.216+0000 7f6954ff9700 1 -- 172.21.15.193:0/1439924779 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f69480cf3a0 con 0x7f6958075420 2024-04-25T13:07:22.216 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.216+0000 7f6956ffd700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f69400a3c40 0x7f69400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:07:22.217 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.216+0000 7f6956ffd700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f69400a3c40 0x7f69400a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f695819c1a0 tx=0x7f694c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:07:22.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.220+0000 7f6954ff9700 1 -- 172.21.15.193:0/1439924779 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6948014030 con 0x7f6958075420 2024-04-25T13:07:22.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:22.424+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6958062e80 con 0x7f69400a3c40 2024-04-25T13:07:23.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:22 smithi072 ceph-mon[29199]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T13:07:23.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:22 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:07:23.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:22 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:07:23.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:22 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:23.036 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:22 smithi193 ceph-mon[32743]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T13:07:23.037 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:22 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:07:23.037 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:22 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:07:23.037 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:22 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:23.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:22 smithi164 ceph-mon[32418]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T13:07:23.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:22 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:07:23.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:22 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:07:23.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:22 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:23.964 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:23 smithi193 ceph-mon[32743]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:07:23.964 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:23 smithi193 ceph-mon[32743]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:24.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:23 smithi072 ceph-mon[29199]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:07:24.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:23 smithi072 ceph-mon[29199]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:24.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:23 smithi164 ceph-mon[32418]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:07:24.055 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:23 smithi164 ceph-mon[32418]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:25.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:25 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/4068835801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c64e72ba-dfe6-43fe-ba1c-4819ac7bc211"}]: dispatch 2024-04-25T13:07:25.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:25 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c64e72ba-dfe6-43fe-ba1c-4819ac7bc211"}]: dispatch 2024-04-25T13:07:25.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:25 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c64e72ba-dfe6-43fe-ba1c-4819ac7bc211"}]': finished 2024-04-25T13:07:25.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:25 smithi193 ceph-mon[32743]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T13:07:25.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:25 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:25 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/4068835801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c64e72ba-dfe6-43fe-ba1c-4819ac7bc211"}]: dispatch 2024-04-25T13:07:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:25 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c64e72ba-dfe6-43fe-ba1c-4819ac7bc211"}]: dispatch 2024-04-25T13:07:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:25 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c64e72ba-dfe6-43fe-ba1c-4819ac7bc211"}]': finished 2024-04-25T13:07:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:25 smithi072 ceph-mon[29199]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T13:07:25.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:25 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:25.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:25 smithi164 ceph-mon[32418]: from='client.? 172.21.15.193:0/4068835801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c64e72ba-dfe6-43fe-ba1c-4819ac7bc211"}]: dispatch 2024-04-25T13:07:25.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:25 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c64e72ba-dfe6-43fe-ba1c-4819ac7bc211"}]: dispatch 2024-04-25T13:07:25.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:25 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c64e72ba-dfe6-43fe-ba1c-4819ac7bc211"}]': finished 2024-04-25T13:07:25.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:25 smithi164 ceph-mon[32418]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T13:07:25.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:25 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:26.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:26 smithi193 ceph-mon[32743]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:26.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:26 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/3850351642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:07:26.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:26 smithi072 ceph-mon[29199]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:26.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:26 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/3850351642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:07:26.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:26 smithi164 ceph-mon[32418]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:26.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:26 smithi164 ceph-mon[32418]: from='client.? 172.21.15.193:0/3850351642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:07:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:28 smithi193 ceph-mon[32743]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:28 smithi072 ceph-mon[29199]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:28.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:28 smithi164 ceph-mon[32418]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:30.305 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:30 smithi164 ceph-mon[32418]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:30.407 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:30 smithi193 ceph-mon[32743]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:30 smithi072 ceph-mon[29199]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:32.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:32 smithi193 ceph-mon[32743]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:32.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:32 smithi072 ceph-mon[29199]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:32 smithi164 ceph-mon[32418]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:33.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:33 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:07:33.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:33 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:33.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:33 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:07:33.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:33 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:33.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:33 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:07:33.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:33 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:34.471 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:34 smithi193 ceph-mon[32743]: Deploying daemon osd.5 on smithi193 2024-04-25T13:07:34.471 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:34 smithi193 ceph-mon[32743]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:34.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:34 smithi072 ceph-mon[29199]: Deploying daemon osd.5 on smithi193 2024-04-25T13:07:34.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:34 smithi072 ceph-mon[29199]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:34.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:34 smithi164 ceph-mon[32418]: Deploying daemon osd.5 on smithi193 2024-04-25T13:07:34.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:34 smithi164 ceph-mon[32418]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:36.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:36 smithi193 ceph-mon[32743]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:36.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:36 smithi072 ceph-mon[29199]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:36.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:36 smithi164 ceph-mon[32418]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:38.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:38 smithi193 ceph-mon[32743]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:38.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:38.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:38 smithi072 ceph-mon[29199]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:38.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:38.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:38.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:38.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:38 smithi164 ceph-mon[32418]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:38.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:38.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:38.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:40.203 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:40 smithi193 ceph-mon[32743]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:40.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:40 smithi072 ceph-mon[29199]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:40.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:40 smithi164 ceph-mon[32418]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:41.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.344+0000 7f6954ff9700 1 -- 172.21.15.193:0/1439924779 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6958062e80 con 0x7f69400a3c40 2024-04-25T13:07:41.343 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-04-25T13:07:41.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f69400a3c40 msgr2=0x7f69400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:41.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f69400a3c40 0x7f69400a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f695819c1a0 tx=0x7f694c009260 comp rx=0 tx=0).stop 2024-04-25T13:07:41.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6958075420 msgr2=0x7f695819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:41.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6958075420 0x7f695819ac90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f694800c9b0 tx=0x7f694800ccc0 comp rx=0 tx=0).stop 2024-04-25T13:07:41.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 shutdown_connections 2024-04-25T13:07:41.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f695810a9a0 0x7f69581a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:41.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6958075420 0x7f695819ac90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:41.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f69400a3c40 0x7f69400a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:41.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 --2- 172.21.15.193:0/1439924779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69580769f0 0x7f695819b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:41.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.346+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 >> 172.21.15.193:0/1439924779 conn(0x7f69580fe040 msgr2=0x7f69581068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:07:41.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.347+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 shutdown_connections 2024-04-25T13:07:41.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:07:41.347+0000 7f695dc13700 1 -- 172.21.15.193:0/1439924779 wait complete. 2024-04-25T13:07:41.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:41.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:41.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:41 smithi193 ceph-mon[32743]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:41.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:41.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:41.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:41.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:41 smithi072 ceph-mon[29199]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:41.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:41 smithi164 ceph-mon[32418]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:41.911 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T13:07:41.914 INFO:tasks.cephadm:Deploying osd.6 on smithi072 with /dev/vg_nvme/lv_2... 2024-04-25T13:07:41.914 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:07:42.160 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:07:42.535 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:42 smithi072 ceph-mon[29199]: from='osd.5 [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:07:42.535 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:42 smithi072 ceph-mon[29199]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:07:42.782 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:42 smithi193 ceph-mon[32743]: from='osd.5 [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:07:42.782 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:42 smithi193 ceph-mon[32743]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:07:42.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:42 smithi164 ceph-mon[32418]: from='osd.5 [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:07:42.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:42 smithi164 ceph-mon[32418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:07:43.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:43 smithi072 ceph-mon[29199]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:07:43.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:43 smithi072 ceph-mon[29199]: from='osd.5 [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:07:43.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:43 smithi072 ceph-mon[29199]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T13:07:43.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:43.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:43 smithi072 ceph-mon[29199]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:07:43.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:43 smithi072 ceph-mon[29199]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:43.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:43 smithi164 ceph-mon[32418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:07:43.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:43 smithi164 ceph-mon[32418]: from='osd.5 [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:07:43.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:43 smithi164 ceph-mon[32418]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T13:07:43.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:43.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:43 smithi164 ceph-mon[32418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:07:43.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:43 smithi164 ceph-mon[32418]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:43.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:43 smithi193 ceph-mon[32743]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:07:43.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:43 smithi193 ceph-mon[32743]: from='osd.5 [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:07:43.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:43 smithi193 ceph-mon[32743]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T13:07:43.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:43.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:43 smithi193 ceph-mon[32743]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:07:43.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:43 smithi193 ceph-mon[32743]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:44.219 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:07:44.219 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:07:44.219 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-04-25T13:07:44.219 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-04-25T13:07:44.219 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113405 s, 925 MB/s 2024-04-25T13:07:44.219 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-04-25T13:07:44.517 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:07:44 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:07:44.432+0000 7f79ebd7c700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T13:07:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:44 smithi072 ceph-mon[29199]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:07:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:44 smithi072 ceph-mon[29199]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:07:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:44.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:44 smithi072 ceph-mon[29199]: from='osd.5 ' entity='osd.5' 2024-04-25T13:07:44.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:44 smithi164 ceph-mon[32418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:07:44.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:44 smithi164 ceph-mon[32418]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:07:44.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:44.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:44.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:44 smithi164 ceph-mon[32418]: from='osd.5 ' entity='osd.5' 2024-04-25T13:07:44.889 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_2 2024-04-25T13:07:44.914 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:07:44 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:07:44.559+0000 7f79e3b67700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:07:44.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:44 smithi193 ceph-mon[32743]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:07:44.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:44 smithi193 ceph-mon[32743]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:07:44.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:44.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:44.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:44 smithi193 ceph-mon[32743]: from='osd.5 ' entity='osd.5' 2024-04-25T13:07:45.147 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:07:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:07:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi193 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi193 to 8544M 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:45.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:45.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:45 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:45.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:07:45.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:07:45.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi193 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi193 to 8544M 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:45.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:45.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:45 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi193 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:07:45.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi193 to 8544M 2024-04-25T13:07:45.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:45.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:45.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:07:45.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:07:45.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:45 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:46.617 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:46 smithi072 ceph-mon[29199]: osd.5 [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] boot 2024-04-25T13:07:46.617 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:46 smithi072 ceph-mon[29199]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T13:07:46.617 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:46.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.803+0000 7fdd60ec8700 1 -- 172.21.15.72:0/2040664478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c1019a0 msgr2=0x7fdd5c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:46.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.803+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/2040664478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c1019a0 0x7fdd5c101dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0099c0 tx=0x7fdd4c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:07:46.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 -- 172.21.15.72:0/2040664478 shutdown_connections 2024-04-25T13:07:46.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/2040664478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c103540 0x7fdd5c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:46.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/2040664478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c1019a0 0x7fdd5c101dc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:46.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/2040664478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c102ba0 0x7fdd5c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:46.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 -- 172.21.15.72:0/2040664478 >> 172.21.15.72:0/2040664478 conn(0x7fdd5c0fcf40 msgr2=0x7fdd5c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:07:46.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 -- 172.21.15.72:0/2040664478 shutdown_connections 2024-04-25T13:07:46.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 -- 172.21.15.72:0/2040664478 wait complete. 2024-04-25T13:07:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 Processor -- start 2024-04-25T13:07:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 -- start start 2024-04-25T13:07:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c1019a0 0x7fdd5c06b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:07:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.804+0000 7fdd60ec8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c102ba0 0x7fdd5c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:07:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd60ec8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c103540 0x7fdd5c1a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:07:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd60ec8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c10d490 con 0x7fdd5c103540 2024-04-25T13:07:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd60ec8700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c10d310 con 0x7fdd5c1019a0 2024-04-25T13:07:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd60ec8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c10d610 con 0x7fdd5c102ba0 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c103540 0x7fdd5c1a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c103540 0x7fdd5c1a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49078/0 (socket says 172.21.15.72:49078) 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5ad9d700 1 -- 172.21.15.72:0/1793355175 learned_addr learned my addr 172.21.15.72:0/1793355175 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5a59c700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c1019a0 0x7fdd5c06b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd59d9b700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c102ba0 0x7fdd5c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5ad9d700 1 -- 172.21.15.72:0/1793355175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c102ba0 msgr2=0x7fdd5c19da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5ad9d700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c102ba0 0x7fdd5c19da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5ad9d700 1 -- 172.21.15.72:0/1793355175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c1019a0 msgr2=0x7fdd5c06b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5ad9d700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c1019a0 0x7fdd5c06b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5ad9d700 1 -- 172.21.15.72:0/1793355175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5c1a0920 con 0x7fdd5c103540 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd5ad9d700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c103540 0x7fdd5c1a0050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007ee0 tx=0x7fdd5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd4b7fe700 1 -- 172.21.15.72:0/1793355175 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54004bb0 con 0x7fdd5c103540 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5c1a0c10 con 0x7fdd5c103540 2024-04-25T13:07:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd4b7fe700 1 -- 172.21.15.72:0/1793355175 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd54004d50 con 0x7fdd5c103540 2024-04-25T13:07:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd4b7fe700 1 -- 172.21.15.72:0/1793355175 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54005920 con 0x7fdd5c103540 2024-04-25T13:07:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.805+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd5c1a1100 con 0x7fdd5c103540 2024-04-25T13:07:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.806+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd5c069730 con 0x7fdd5c103540 2024-04-25T13:07:46.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:46 smithi164 ceph-mon[32418]: osd.5 [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] boot 2024-04-25T13:07:46.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:46 smithi164 ceph-mon[32418]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T13:07:46.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:46.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.807+0000 7fdd4b7fe700 1 -- 172.21.15.72:0/1793355175 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd5401a020 con 0x7fdd5c103540 2024-04-25T13:07:46.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.808+0000 7fdd4b7fe700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdd440a3c40 0x7fdd440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:07:46.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.808+0000 7fdd5a59c700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdd440a3c40 0x7fdd440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:07:46.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.808+0000 7fdd4b7fe700 1 -- 172.21.15.72:0/1793355175 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fdd540cecb0 con 0x7fdd5c103540 2024-04-25T13:07:46.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.808+0000 7fdd5a59c700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdd440a3c40 0x7fdd440a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c006010 tx=0x7fdd4c003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:07:46.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:46.813+0000 7fdd4b7fe700 1 -- 172.21.15.72:0/1793355175 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd54010070 con 0x7fdd5c103540 2024-04-25T13:07:46.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:46 smithi193 ceph-mon[32743]: osd.5 [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] boot 2024-04-25T13:07:46.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:46 smithi193 ceph-mon[32743]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T13:07:46.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:07:47.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:07:47.020+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdd5c062e80 con 0x7fdd440a3c40 2024-04-25T13:07:47.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:47 smithi164 ceph-mon[32418]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:07:47.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:47 smithi164 ceph-mon[32418]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:07:47.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:47 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:07:47.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:47 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:07:47.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:47 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:47.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:47 smithi164 ceph-mon[32418]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:07:47.880 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:47 smithi072 ceph-mon[29199]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:07:47.880 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:47 smithi072 ceph-mon[29199]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:07:47.881 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:07:47.881 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:07:47.881 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:47.881 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:47 smithi072 ceph-mon[29199]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:07:47.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:47 smithi193 ceph-mon[32743]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:07:47.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:47 smithi193 ceph-mon[32743]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:07:47.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:47 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:07:47.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:47 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:07:47.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:47 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:47.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:47 smithi193 ceph-mon[32743]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:07:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:48 smithi164 ceph-mon[32418]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T13:07:48.881 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:48 smithi072 ceph-mon[29199]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T13:07:48.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:48 smithi193 ceph-mon[32743]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T13:07:49.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:49 smithi164 ceph-mon[32418]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T13:07:49.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:49 smithi164 ceph-mon[32418]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:07:49.910 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:49 smithi072 ceph-mon[29199]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T13:07:49.910 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:49 smithi072 ceph-mon[29199]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:07:49.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:49 smithi193 ceph-mon[32743]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T13:07:49.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:49 smithi193 ceph-mon[32743]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:07:50.914 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:50 smithi164 ceph-mon[32418]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T13:07:50.914 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:50 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3462690810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd"}]: dispatch 2024-04-25T13:07:50.915 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:50 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3462690810' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd"}]': finished 2024-04-25T13:07:50.915 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:50 smithi164 ceph-mon[32418]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T13:07:50.915 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:07:50.915 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:50 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1995189242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:07:50.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:50 smithi193 ceph-mon[32743]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T13:07:50.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:50 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3462690810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd"}]: dispatch 2024-04-25T13:07:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:50 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3462690810' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd"}]': finished 2024-04-25T13:07:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:50 smithi193 ceph-mon[32743]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T13:07:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:07:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:50 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1995189242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:07:51.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:50 smithi072 ceph-mon[29199]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T13:07:51.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:50 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3462690810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd"}]: dispatch 2024-04-25T13:07:51.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:50 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3462690810' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd"}]': finished 2024-04-25T13:07:51.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:50 smithi072 ceph-mon[29199]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T13:07:51.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:07:51.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:50 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1995189242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:07:51.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:51 smithi193 ceph-mon[32743]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:07:52.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:51 smithi072 ceph-mon[29199]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:07:52.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:51 smithi164 ceph-mon[32418]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:07:54.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:54 smithi072 ceph-mon[29199]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T13:07:54.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:54 smithi193 ceph-mon[32743]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T13:07:54.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:54 smithi164 ceph-mon[32418]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T13:07:56.366 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:56 smithi072 ceph-mon[29199]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T13:07:56.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:56 smithi193 ceph-mon[32743]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T13:07:56.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:56 smithi164 ceph-mon[32418]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T13:07:58.412 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:58 smithi072 ceph-mon[29199]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:07:58.413 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:58 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:07:58.413 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:58 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:58.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:58 smithi193 ceph-mon[32743]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:07:58.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:58 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:07:58.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:58 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:58 smithi164 ceph-mon[32418]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:07:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:58 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:07:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:58 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:59.461 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:07:59 smithi072 ceph-mon[29199]: Deploying daemon osd.6 on smithi072 2024-04-25T13:07:59.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:07:59 smithi193 ceph-mon[32743]: Deploying daemon osd.6 on smithi072 2024-04-25T13:07:59.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:07:59 smithi164 ceph-mon[32418]: Deploying daemon osd.6 on smithi072 2024-04-25T13:08:00.420 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:00 smithi164 ceph-mon[32418]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:08:00.475 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:00 smithi072 ceph-mon[29199]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:08:00.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:00 smithi193 ceph-mon[32743]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:08:02.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:02 smithi072 ceph-mon[29199]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-25T13:08:02.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:02 smithi193 ceph-mon[32743]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-25T13:08:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:02 smithi164 ceph-mon[32418]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-25T13:08:04.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:04 smithi072 ceph-mon[29199]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:04.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:04 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:04.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:04 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:04.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:04 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:04.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:04 smithi193 ceph-mon[32743]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:04.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:04 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:04.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:04 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:04.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:04 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:04.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:04 smithi164 ceph-mon[32418]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:04.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:04 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:04.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:04 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:04.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:04 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:06.654 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:06 smithi072 ceph-mon[29199]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:06.712 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 6 on host 'smithi072' 2024-04-25T13:08:06.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.712+0000 7fdd4b7fe700 1 -- 172.21.15.72:0/1793355175 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdd5c062e80 con 0x7fdd440a3c40 2024-04-25T13:08:06.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdd440a3c40 msgr2=0x7fdd440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdd440a3c40 0x7fdd440a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c006010 tx=0x7fdd4c003720 comp rx=0 tx=0).stop 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c103540 msgr2=0x7fdd5c1a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c103540 0x7fdd5c1a0050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007ee0 tx=0x7fdd5400d3b0 comp rx=0 tx=0).stop 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 shutdown_connections 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fdd440a3c40 0x7fdd440a6100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd5c103540 0x7fdd5c1a0050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c1019a0 0x7fdd5c06b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 --2- 172.21.15.72:0/1793355175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c102ba0 0x7fdd5c19da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 >> 172.21.15.72:0/1793355175 conn(0x7fdd5c0fcf40 msgr2=0x7fdd5c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 shutdown_connections 2024-04-25T13:08:06.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:08:06.715+0000 7fdd60ec8700 1 -- 172.21.15.72:0/1793355175 wait complete. 2024-04-25T13:08:06.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:06 smithi193 ceph-mon[32743]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:06.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:06 smithi164 ceph-mon[32418]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:07.340 DEBUG:teuthology.orchestra.run.smithi072:osd.6> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T13:08:07.343 INFO:tasks.cephadm:Deploying osd.7 on smithi164 with /dev/vg_nvme/lv_2... 2024-04-25T13:08:07.343 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:08:07.596 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:08:07.944 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:07.944 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:07.944 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:07 smithi164 ceph-mon[32418]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:07.944 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:07 smithi164 ceph-mon[32418]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:08:07.944 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:07.944 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:07.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:07.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:07.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:07 smithi193 ceph-mon[32743]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:07.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:07 smithi193 ceph-mon[32743]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:08:07.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:07.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:08.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:08.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:08.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:07 smithi072 ceph-mon[29199]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:08.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:07 smithi072 ceph-mon[29199]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:08:08.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:08.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:08.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:08 smithi193 ceph-mon[32743]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:08:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:08 smithi193 ceph-mon[32743]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T13:08:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:08 smithi193 ceph-mon[32743]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:08:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:09.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:08 smithi072 ceph-mon[29199]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:08:09.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:08 smithi072 ceph-mon[29199]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T13:08:09.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:08 smithi072 ceph-mon[29199]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:08:09.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:09.048 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:08 smithi164 ceph-mon[32418]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:08:09.048 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:08 smithi164 ceph-mon[32418]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T13:08:09.048 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:08 smithi164 ceph-mon[32418]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:08:09.048 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:09.718 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:08:09.718 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:08:09.718 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-25T13:08:09.719 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-25T13:08:09.719 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0148226 s, 707 MB/s 2024-04-25T13:08:09.719 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-25T13:08:09.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:09 smithi193 ceph-mon[32743]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:08:09.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:09 smithi193 ceph-mon[32743]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T13:08:09.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:09 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:09.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:09 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:09.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:09 smithi193 ceph-mon[32743]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:09.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:09 smithi193 ceph-mon[32743]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' 2024-04-25T13:08:10.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:09 smithi072 ceph-mon[29199]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:08:10.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:09 smithi072 ceph-mon[29199]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T13:08:10.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:10.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:09 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:10.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:09 smithi072 ceph-mon[29199]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:10.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:09 smithi072 ceph-mon[29199]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' 2024-04-25T13:08:10.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:08:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:08:09.648+0000 7f5c969db700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T13:08:10.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:08:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:08:09.831+0000 7f5c8e7c6700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:08:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:09 smithi164 ceph-mon[32418]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:08:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:09 smithi164 ceph-mon[32418]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T13:08:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:09 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:09 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:10.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:09 smithi164 ceph-mon[32418]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:10.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:09 smithi164 ceph-mon[32418]: from='osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765]' entity='osd.6' 2024-04-25T13:08:10.314 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2024-04-25T13:08:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:08:10.843 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:10 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:08:10.843 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:10 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:08:10.844 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:10 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:10.844 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:10 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:10.965 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:10 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:08:10.965 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:10 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:08:10.965 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:10 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:10.965 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:10 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:10.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:10 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:08:10.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:10 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:08:10.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:10 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:10.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:10 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] boot 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi072 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:11.955 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi072 to 5696M 2024-04-25T13:08:11.956 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:11.956 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:11.956 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:11.956 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:11.956 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:11.956 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:11.956 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:11.956 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:11.956 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:11 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:11.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] boot 2024-04-25T13:08:11.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi072 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi072 to 5696M 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:11.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:11.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:11.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:11.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:11 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:12.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: osd.6 [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] boot 2024-04-25T13:08:12.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T13:08:12.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:08:12.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi072 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi072 to 5696M 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:12.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:12.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:11 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:12.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 -- 172.21.15.164:0/2984146428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4102b90 msgr2=0x7f74a4102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:12.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 --2- 172.21.15.164:0/2984146428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4102b90 0x7f74a4102ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f748c0099c0 tx=0x7f748c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:08:12.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 -- 172.21.15.164:0/2984146428 shutdown_connections 2024-04-25T13:08:12.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 --2- 172.21.15.164:0/2984146428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74a4103530 0x7f74a4109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:12.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 --2- 172.21.15.164:0/2984146428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a4101990 0x7f74a4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:12.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 --2- 172.21.15.164:0/2984146428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4102b90 0x7f74a4102ff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:12.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 -- 172.21.15.164:0/2984146428 >> 172.21.15.164:0/2984146428 conn(0x7f74a40fcf30 msgr2=0x7f74a40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:08:12.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 -- 172.21.15.164:0/2984146428 shutdown_connections 2024-04-25T13:08:12.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 -- 172.21.15.164:0/2984146428 wait complete. 2024-04-25T13:08:12.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.175+0000 7f74a9051700 1 Processor -- start 2024-04-25T13:08:12.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a9051700 1 -- start start 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a9051700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74a4101990 0x7f74a410c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a9051700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a4102b90 0x7f74a410a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a9051700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4103530 0x7f74a410aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a9051700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f74a410e5a0 con 0x7f74a4101990 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a9051700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74a410e420 con 0x7f74a4103530 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a9051700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74a410e720 con 0x7f74a4102b90 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a359e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4103530 0x7f74a410aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a359e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4103530 0x7f74a410aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45650/0 (socket says 172.21.15.164:45650) 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a359e700 1 -- 172.21.15.164:0/4116426088 learned_addr learned my addr 172.21.15.164:0/4116426088 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f749a59c700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a4102b90 0x7f74a410a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a2d9d700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74a4101990 0x7f74a410c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a359e700 1 -- 172.21.15.164:0/4116426088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a4102b90 msgr2=0x7f74a410a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a359e700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a4102b90 0x7f74a410a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a359e700 1 -- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74a4101990 msgr2=0x7f74a410c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a359e700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74a4101990 0x7f74a410c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a359e700 1 -- 172.21.15.164:0/4116426088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a410b7a0 con 0x7f74a4103530 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.176+0000 7f74a2d9d700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74a4101990 0x7f74a410c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.177+0000 7f74a359e700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4103530 0x7f74a410aef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c900 tx=0x7f749c00cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.177+0000 7f74a0d99700 1 -- 172.21.15.164:0/4116426088 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c007b00 con 0x7f74a4103530 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.177+0000 7f74a0d99700 1 -- 172.21.15.164:0/4116426088 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f749c00ce50 con 0x7f74a4103530 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.177+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a41a2d10 con 0x7f74a4103530 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.177+0000 7f74a0d99700 1 -- 172.21.15.164:0/4116426088 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c018660 con 0x7f74a4103530 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.177+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a41a31c0 con 0x7f74a4103530 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.178+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a4069730 con 0x7f74a4103530 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.179+0000 7f74a0d99700 1 -- 172.21.15.164:0/4116426088 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f749c00f450 con 0x7f74a4103530 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.180+0000 7f74a0d99700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f74900a3cc0 0x7f74900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.180+0000 7f74a0d99700 1 -- 172.21.15.164:0/4116426088 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f749c0cf590 con 0x7f74a4103530 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.180+0000 7f74a2d9d700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f74900a3cc0 0x7f74900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.181+0000 7f74a2d9d700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f74900a3cc0 0x7f74900a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7494007fd0 tx=0x7f7494007830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.184+0000 7f74a0d99700 1 -- 172.21.15.164:0/4116426088 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f749c014030 con 0x7f74a4103530 2024-04-25T13:08:12.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:12.391+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f74a4062e80 con 0x7f74900a3cc0 2024-04-25T13:08:13.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:13 smithi164 ceph-mon[32418]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T13:08:13.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:13 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:08:13.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:13 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:08:13.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:13 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:13.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:13 smithi193 ceph-mon[32743]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T13:08:13.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:13 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:08:13.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:13 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:08:13.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:13 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:13.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:13 smithi072 ceph-mon[29199]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T13:08:13.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:13 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:08:13.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:13 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:08:13.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:13 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:14.222 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:14 smithi164 ceph-mon[32418]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:08:14.222 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:14 smithi164 ceph-mon[32418]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:14 smithi193 ceph-mon[32743]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:08:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:14 smithi193 ceph-mon[32743]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:14 smithi072 ceph-mon[29199]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:08:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:14 smithi072 ceph-mon[29199]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:15.395 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:15 smithi164 ceph-mon[32418]: from='client.? 172.21.15.164:0/560211623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca180ab-29ed-4109-bb1d-f7d19cb086b5"}]: dispatch 2024-04-25T13:08:15.396 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:15 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca180ab-29ed-4109-bb1d-f7d19cb086b5"}]: dispatch 2024-04-25T13:08:15.396 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:15 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ca180ab-29ed-4109-bb1d-f7d19cb086b5"}]': finished 2024-04-25T13:08:15.396 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:15 smithi164 ceph-mon[32418]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T13:08:15.396 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:15 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:15.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:15 smithi193 ceph-mon[32743]: from='client.? 172.21.15.164:0/560211623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca180ab-29ed-4109-bb1d-f7d19cb086b5"}]: dispatch 2024-04-25T13:08:15.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:15 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca180ab-29ed-4109-bb1d-f7d19cb086b5"}]: dispatch 2024-04-25T13:08:15.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:15 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ca180ab-29ed-4109-bb1d-f7d19cb086b5"}]': finished 2024-04-25T13:08:15.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:15 smithi193 ceph-mon[32743]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T13:08:15.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:15 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:15.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:15 smithi072 ceph-mon[29199]: from='client.? 172.21.15.164:0/560211623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca180ab-29ed-4109-bb1d-f7d19cb086b5"}]: dispatch 2024-04-25T13:08:15.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:15 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ca180ab-29ed-4109-bb1d-f7d19cb086b5"}]: dispatch 2024-04-25T13:08:15.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:15 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ca180ab-29ed-4109-bb1d-f7d19cb086b5"}]': finished 2024-04-25T13:08:15.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:15 smithi072 ceph-mon[29199]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T13:08:15.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:15 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:16.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:16 smithi193 ceph-mon[32743]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:16.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:16 smithi193 ceph-mon[32743]: from='client.? 172.21.15.164:0/1200408449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:08:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:16 smithi072 ceph-mon[29199]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:16 smithi072 ceph-mon[29199]: from='client.? 172.21.15.164:0/1200408449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:08:16.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:16 smithi164 ceph-mon[32418]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:16.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:16 smithi164 ceph-mon[32418]: from='client.? 172.21.15.164:0/1200408449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:08:18.367 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:18 smithi164 ceph-mon[32418]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:18.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:18 smithi193 ceph-mon[32743]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:18.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:18 smithi072 ceph-mon[29199]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:20.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:20 smithi164 ceph-mon[32418]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:20.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:20 smithi193 ceph-mon[32743]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:20.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:20 smithi072 ceph-mon[29199]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:22.372 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:22 smithi164 ceph-mon[32418]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:22.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:22 smithi193 ceph-mon[32743]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:22.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:22 smithi072 ceph-mon[29199]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:23.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:08:23.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:23.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:08:23.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:08:23.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:24.381 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:24 smithi164 ceph-mon[32418]: Deploying daemon osd.7 on smithi164 2024-04-25T13:08:24.382 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:24 smithi164 ceph-mon[32418]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:24.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:24 smithi193 ceph-mon[32743]: Deploying daemon osd.7 on smithi164 2024-04-25T13:08:24.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:24 smithi193 ceph-mon[32743]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:24.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:24 smithi072 ceph-mon[29199]: Deploying daemon osd.7 on smithi164 2024-04-25T13:08:24.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:24 smithi072 ceph-mon[29199]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:26.214 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:26 smithi164 ceph-mon[32418]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:26.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:26 smithi193 ceph-mon[32743]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:26.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:26 smithi072 ceph-mon[29199]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:28.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:28 smithi164 ceph-mon[32418]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:28.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:28 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:28.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:28 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:28.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:28 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:28 smithi193 ceph-mon[32743]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:28 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:28.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:28 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:28 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:28 smithi072 ceph-mon[29199]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:28 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:28 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:28 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:30.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:30 smithi164 ceph-mon[32418]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:30.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:30 smithi193 ceph-mon[32743]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:30 smithi072 ceph-mon[29199]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.655+0000 7f74a0d99700 1 -- 172.21.15.164:0/4116426088 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f74a4062e80 con 0x7f74900a3cc0 2024-04-25T13:08:31.655 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 7 on host 'smithi164' 2024-04-25T13:08:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f74900a3cc0 msgr2=0x7f74900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f74900a3cc0 0x7f74900a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7494007fd0 tx=0x7f7494007830 comp rx=0 tx=0).stop 2024-04-25T13:08:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4103530 msgr2=0x7f74a410aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4103530 0x7f74a410aef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c900 tx=0x7f749c00cc10 comp rx=0 tx=0).stop 2024-04-25T13:08:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 shutdown_connections 2024-04-25T13:08:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a4103530 0x7f74a410aef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f74900a3cc0 0x7f74900a6180 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74a4101990 0x7f74a410c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 --2- 172.21.15.164:0/4116426088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a4102b90 0x7f74a410a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:31.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 >> 172.21.15.164:0/4116426088 conn(0x7f74a40fcf30 msgr2=0x7f74a4105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:08:31.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 shutdown_connections 2024-04-25T13:08:31.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:08:31.658+0000 7f74a9051700 1 -- 172.21.15.164:0/4116426088 wait complete. 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:31.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='osd.7 [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:08:31.975 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:31 smithi164 ceph-mon[32418]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:08:32.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='osd.7 [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:08:32.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:31 smithi193 ceph-mon[32743]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:08:32.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:32.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:32.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:32.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:32.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:32.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='osd.7 [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:08:32.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:31 smithi072 ceph-mon[29199]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:08:32.283 DEBUG:teuthology.orchestra.run.smithi164:osd.7> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T13:08:32.286 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2024-04-25T13:08:32.286 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:08:32.531 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:08:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:33 smithi164 ceph-mon[32418]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:08:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:33 smithi164 ceph-mon[32418]: from='osd.7 [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:08:33.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:33 smithi164 ceph-mon[32418]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:08:33.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:33 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:33.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:33 smithi164 ceph-mon[32418]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:08:33.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:33 smithi164 ceph-mon[32418]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:33.807 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:08:33 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:08:33.620+0000 7f76b7294700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T13:08:33.807 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:08:33 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:08:33.631+0000 7f76b3087700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:08:33.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:33 smithi193 ceph-mon[32743]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:08:33.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:33 smithi193 ceph-mon[32743]: from='osd.7 [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:08:33.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:33 smithi193 ceph-mon[32743]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:08:33.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:33 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:33.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:33 smithi193 ceph-mon[32743]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:08:33.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:33 smithi193 ceph-mon[32743]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:34.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:33 smithi072 ceph-mon[29199]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:08:34.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:33 smithi072 ceph-mon[29199]: from='osd.7 [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:08:34.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:33 smithi072 ceph-mon[29199]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:08:34.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:33 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:34.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:33 smithi072 ceph-mon[29199]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:08:34.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:33 smithi072 ceph-mon[29199]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:08:34.575 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:08:34.575 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:08:34.575 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-04-25T13:08:34.575 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-04-25T13:08:34.575 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113327 s, 925 MB/s 2024-04-25T13:08:34.575 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-04-25T13:08:34.756 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:34 smithi193 ceph-mon[32743]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:08:34.756 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:34 smithi193 ceph-mon[32743]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:08:34.757 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:34 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:34.757 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:34 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:34.757 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:34 smithi193 ceph-mon[32743]: osd.7 [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] boot 2024-04-25T13:08:34.757 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:34 smithi193 ceph-mon[32743]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T13:08:34.757 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:34 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:34.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:34 smithi164 ceph-mon[32418]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:08:34.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:34 smithi164 ceph-mon[32418]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:08:34.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:34 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:34.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:34 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:34.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:34 smithi164 ceph-mon[32418]: osd.7 [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] boot 2024-04-25T13:08:34.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:34 smithi164 ceph-mon[32418]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T13:08:34.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:34 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:34 smithi072 ceph-mon[29199]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:08:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:34 smithi072 ceph-mon[29199]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:08:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:34 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:35.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:34 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:35.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:34 smithi072 ceph-mon[29199]: osd.7 [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] boot 2024-04-25T13:08:35.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:34 smithi072 ceph-mon[29199]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T13:08:35.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:34 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:08:35.228 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-04-25T13:08:35.475 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:08:35.902 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi164 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi164 to 5696M 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:35.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:36.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:08:36.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi164 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi164 to 5696M 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:36.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:08:36.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:08:36.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:36.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi164 2024-04-25T13:08:36.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:36.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:08:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi164 to 5696M 2024-04-25T13:08:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:36.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:36 smithi193 ceph-mon[32743]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T13:08:36.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 -- 172.21.15.193:0/4271084163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc103540 msgr2=0x7f06cc109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:36.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 --2- 172.21.15.193:0/4271084163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc103540 0x7f06cc109df0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06c400b0a0 tx=0x7f06c400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:08:36.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 -- 172.21.15.193:0/4271084163 shutdown_connections 2024-04-25T13:08:36.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 --2- 172.21.15.193:0/4271084163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc103540 0x7f06cc109df0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:36.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 --2- 172.21.15.193:0/4271084163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06cc1019a0 0x7f06cc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:36.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 --2- 172.21.15.193:0/4271084163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06cc102ba0 0x7f06cc103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:36.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 -- 172.21.15.193:0/4271084163 >> 172.21.15.193:0/4271084163 conn(0x7f06cc0fcf40 msgr2=0x7f06cc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 -- 172.21.15.193:0/4271084163 shutdown_connections 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 -- 172.21.15.193:0/4271084163 wait complete. 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 Processor -- start 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.963+0000 7f06d329b700 1 -- start start 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d329b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc1019a0 0x7f06cc105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d329b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06cc102ba0 0x7f06cc103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d329b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06cc103540 0x7f06cc1044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d329b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc10d490 con 0x7f06cc102ba0 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d329b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc10d310 con 0x7f06cc103540 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d329b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc10d610 con 0x7f06cc1019a0 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1037700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc1019a0 0x7f06cc105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1037700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc1019a0 0x7f06cc105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:53630/0 (socket says 172.21.15.193:53630) 2024-04-25T13:08:36.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1037700 1 -- 172.21.15.193:0/1705231459 learned_addr learned my addr 172.21.15.193:0/1705231459 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1838700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06cc103540 0x7f06cc1044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d0836700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06cc102ba0 0x7f06cc103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1037700 1 -- 172.21.15.193:0/1705231459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06cc103540 msgr2=0x7f06cc1044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1037700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06cc103540 0x7f06cc1044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1037700 1 -- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06cc102ba0 msgr2=0x7f06cc103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1037700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06cc102ba0 0x7f06cc103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1037700 1 -- 172.21.15.193:0/1705231459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc104d70 con 0x7f06cc1019a0 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d0836700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06cc102ba0 0x7f06cc103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d1037700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc1019a0 0x7f06cc105930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f06c800c990 tx=0x7f06c800cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06c27fc700 1 -- 172.21.15.193:0/1705231459 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8007b00 con 0x7f06cc1019a0 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06c27fc700 1 -- 172.21.15.193:0/1705231459 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c8007ca0 con 0x7f06cc1019a0 2024-04-25T13:08:36.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06c27fc700 1 -- 172.21.15.193:0/1705231459 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8018660 con 0x7f06cc1019a0 2024-04-25T13:08:36.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.964+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc1a2c90 con 0x7f06cc1019a0 2024-04-25T13:08:36.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.965+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc1a3180 con 0x7f06cc1019a0 2024-04-25T13:08:36.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.965+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc069760 con 0x7f06cc1019a0 2024-04-25T13:08:36.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.967+0000 7f06c27fc700 1 -- 172.21.15.193:0/1705231459 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f06c8004030 con 0x7f06cc1019a0 2024-04-25T13:08:36.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.967+0000 7f06c27fc700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f06b80a3cc0 0x7f06b80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:08:36.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.967+0000 7f06c27fc700 1 -- 172.21.15.193:0/1705231459 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f06c80cf8c0 con 0x7f06cc1019a0 2024-04-25T13:08:36.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.968+0000 7f06d0836700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f06b80a3cc0 0x7f06b80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:08:36.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.968+0000 7f06d0836700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f06b80a3cc0 0x7f06b80a6180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06bc007900 tx=0x7f06bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:08:36.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:36.971+0000 7f06c27fc700 1 -- 172.21.15.193:0/1705231459 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06cc069760 con 0x7f06cc1019a0 2024-04-25T13:08:37.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:36 smithi072 ceph-mon[29199]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T13:08:37.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:36 smithi164 ceph-mon[32418]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T13:08:37.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:37.179+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f06cc0ff070 con 0x7f06b80a3cc0 2024-04-25T13:08:37.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:37 smithi193 ceph-mon[32743]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:08:37.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:37 smithi193 ceph-mon[32743]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:37.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:37 smithi193 ceph-mon[32743]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:08:37.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:37 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:08:37.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:37 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:08:37.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:37 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:37.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:37 smithi193 ceph-mon[32743]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:08:38.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:37 smithi072 ceph-mon[29199]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:08:38.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:37 smithi072 ceph-mon[29199]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:38.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:37 smithi072 ceph-mon[29199]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:08:38.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:37 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:08:38.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:37 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:08:38.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:37 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:38.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:37 smithi072 ceph-mon[29199]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:08:38.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:37 smithi164 ceph-mon[32418]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:08:38.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:37 smithi164 ceph-mon[32418]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:38.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:37 smithi164 ceph-mon[32418]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:08:38.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:37 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:08:38.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:37 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:08:38.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:37 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:38.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:37 smithi164 ceph-mon[32418]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:08:39.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:39 smithi193 ceph-mon[32743]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:08:39.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:39 smithi072 ceph-mon[29199]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:08:39.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:39 smithi164 ceph-mon[32418]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:08:40.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:40 smithi072 ceph-mon[29199]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:40.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:40 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/2238108678' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd29481d-1fbb-4b9b-b066-f85db8310162"}]: dispatch 2024-04-25T13:08:40.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:40 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd29481d-1fbb-4b9b-b066-f85db8310162"}]: dispatch 2024-04-25T13:08:40.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:40 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd29481d-1fbb-4b9b-b066-f85db8310162"}]': finished 2024-04-25T13:08:40.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:40 smithi072 ceph-mon[29199]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T13:08:40.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:40 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:40.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:40 smithi164 ceph-mon[32418]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:40.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:40 smithi164 ceph-mon[32418]: from='client.? 172.21.15.193:0/2238108678' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd29481d-1fbb-4b9b-b066-f85db8310162"}]: dispatch 2024-04-25T13:08:40.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:40 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd29481d-1fbb-4b9b-b066-f85db8310162"}]: dispatch 2024-04-25T13:08:40.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:40 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd29481d-1fbb-4b9b-b066-f85db8310162"}]': finished 2024-04-25T13:08:40.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:40 smithi164 ceph-mon[32418]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T13:08:40.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:40 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:40 smithi193 ceph-mon[32743]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:40 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/2238108678' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd29481d-1fbb-4b9b-b066-f85db8310162"}]: dispatch 2024-04-25T13:08:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:40 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd29481d-1fbb-4b9b-b066-f85db8310162"}]: dispatch 2024-04-25T13:08:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:40 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd29481d-1fbb-4b9b-b066-f85db8310162"}]': finished 2024-04-25T13:08:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:40 smithi193 ceph-mon[32743]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T13:08:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:40 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:41.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:41 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/3177475441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:08:41.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:41 smithi164 ceph-mon[32418]: from='client.? 172.21.15.193:0/3177475441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:08:41.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:41 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/3177475441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:08:42.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:42 smithi072 ceph-mon[29199]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:42.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:42 smithi164 ceph-mon[32418]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:42.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:42 smithi193 ceph-mon[32743]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:44.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:44 smithi193 ceph-mon[32743]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:44.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:44 smithi072 ceph-mon[29199]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:44.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:44 smithi164 ceph-mon[32418]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:46.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:46 smithi193 ceph-mon[32743]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:46.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:46 smithi072 ceph-mon[29199]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:46.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:46 smithi164 ceph-mon[32418]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:48.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:48 smithi193 ceph-mon[32743]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:48.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:08:48.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:48.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:48 smithi193 ceph-mon[32743]: Deploying daemon osd.8 on smithi193 2024-04-25T13:08:48.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:48 smithi072 ceph-mon[29199]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:48.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:08:48.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:48.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:48 smithi072 ceph-mon[29199]: Deploying daemon osd.8 on smithi193 2024-04-25T13:08:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:48 smithi164 ceph-mon[32418]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:08:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:48 smithi164 ceph-mon[32418]: Deploying daemon osd.8 on smithi193 2024-04-25T13:08:50.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:50 smithi193 ceph-mon[32743]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:50.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:50 smithi072 ceph-mon[29199]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:50.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:50 smithi164 ceph-mon[32418]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:52.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:52 smithi193 ceph-mon[32743]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:52.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:52 smithi072 ceph-mon[29199]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:52.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:52 smithi164 ceph-mon[32418]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:53.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:53.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:53.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:53.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:53 smithi164 ceph-mon[32418]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:53 smithi193 ceph-mon[32743]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:54.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:54.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:54.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:54.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:53 smithi072 ceph-mon[29199]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:56.100 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.100+0000 7f06c27fc700 1 -- 172.21.15.193:0/1705231459 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f06cc0ff070 con 0x7f06b80a3cc0 2024-04-25T13:08:56.100 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2024-04-25T13:08:56.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f06b80a3cc0 msgr2=0x7f06b80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:56.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f06b80a3cc0 0x7f06b80a6180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06bc007900 tx=0x7f06bc008040 comp rx=0 tx=0).stop 2024-04-25T13:08:56.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc1019a0 msgr2=0x7f06cc105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:08:56.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc1019a0 0x7f06cc105930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f06c800c990 tx=0x7f06c800cca0 comp rx=0 tx=0).stop 2024-04-25T13:08:56.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 shutdown_connections 2024-04-25T13:08:56.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06cc103540 0x7f06cc1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:56.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06cc1019a0 0x7f06cc105930 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:56.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f06b80a3cc0 0x7f06b80a6180 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:56.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 --2- 172.21.15.193:0/1705231459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06cc102ba0 0x7f06cc103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:08:56.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 >> 172.21.15.193:0/1705231459 conn(0x7f06cc0fcf40 msgr2=0x7f06cc0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:08:56.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 shutdown_connections 2024-04-25T13:08:56.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:08:56.102+0000 7f06d329b700 1 -- 172.21.15.193:0/1705231459 wait complete. 2024-04-25T13:08:56.390 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:56 smithi193 ceph-mon[32743]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:56.390 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.390 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.390 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:56.390 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:56.390 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.390 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.390 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.390 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:56 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:56 smithi072 ceph-mon[29199]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:56.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:56.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:56 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:56.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:56 smithi164 ceph-mon[32418]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:56.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:56.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:56.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:08:56.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:56 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:56.662 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T13:08:56.664 INFO:tasks.cephadm:Deploying osd.9 on smithi072 with /dev/vg_nvme/lv_1... 2024-04-25T13:08:56.664 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:08:56.909 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:08:57.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:57 smithi193 ceph-mon[32743]: from='osd.8 [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:08:57.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:57 smithi193 ceph-mon[32743]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:08:57.266 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:57 smithi072 ceph-mon[29199]: from='osd.8 [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:08:57.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:57 smithi072 ceph-mon[29199]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:08:57.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:57 smithi164 ceph-mon[32418]: from='osd.8 [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:08:57.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:57 smithi164 ceph-mon[32418]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:08:58.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:58 smithi193 ceph-mon[32743]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:58.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:58 smithi193 ceph-mon[32743]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:08:58.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:58 smithi193 ceph-mon[32743]: from='osd.8 [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:08:58.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:58 smithi193 ceph-mon[32743]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T13:08:58.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:58 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:58.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:58 smithi193 ceph-mon[32743]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:08:58.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:58 smithi072 ceph-mon[29199]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:58.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:58 smithi072 ceph-mon[29199]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:08:58.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:58 smithi072 ceph-mon[29199]: from='osd.8 [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:08:58.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:58 smithi072 ceph-mon[29199]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T13:08:58.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:58 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:58.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:58 smithi072 ceph-mon[29199]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:08:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:58 smithi164 ceph-mon[32418]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:08:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:58 smithi164 ceph-mon[32418]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:08:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:58 smithi164 ceph-mon[32418]: from='osd.8 [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:08:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:58 smithi164 ceph-mon[32418]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T13:08:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:58 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:58 smithi164 ceph-mon[32418]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:08:58.918 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:08:58.919 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:08:58.919 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-04-25T13:08:58.919 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-04-25T13:08:58.919 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115616 s, 907 MB/s 2024-04-25T13:08:58.919 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-04-25T13:08:59.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:59 smithi193 ceph-mon[32743]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:08:59.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:59 smithi193 ceph-mon[32743]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:08:59.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:59.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:59.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:59 smithi193 ceph-mon[32743]: from='osd.8 ' entity='osd.8' 2024-04-25T13:08:59.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:59 smithi193 ceph-mon[32743]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T13:08:59.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:08:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:59.231 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:08:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:08:59.087+0000 7fde33114700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T13:08:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:59 smithi072 ceph-mon[29199]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:08:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:59 smithi072 ceph-mon[29199]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:08:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:59 smithi072 ceph-mon[29199]: from='osd.8 ' entity='osd.8' 2024-04-25T13:08:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:59 smithi072 ceph-mon[29199]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T13:08:59.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:08:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:59 smithi164 ceph-mon[32418]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:08:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:59 smithi164 ceph-mon[32418]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:08:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:59 smithi164 ceph-mon[32418]: from='osd.8 ' entity='osd.8' 2024-04-25T13:08:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:59 smithi164 ceph-mon[32418]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T13:08:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:08:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:08:59.617 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_1 2024-04-25T13:08:59.662 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:08:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:08:59.238+0000 7fde2aeff700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:08:59.865 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:00.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:00.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: osd.8 [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] boot 2024-04-25T13:09:00.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T13:09:00.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:00.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: osd.8 [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] boot 2024-04-25T13:09:00.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T13:09:00.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:09:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:09:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:09:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:09:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:09:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:00.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:00.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:00.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:00.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:00.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:00.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: osd.8 [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] boot 2024-04-25T13:09:00.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T13:09:00.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:09:01.280 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:01 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi193 2024-04-25T13:09:01.280 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:01 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi193 to 5696M 2024-04-25T13:09:01.280 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:01 smithi072 ceph-mon[29199]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T13:09:01.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:01 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi193 2024-04-25T13:09:01.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:01 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi193 to 5696M 2024-04-25T13:09:01.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:01 smithi193 ceph-mon[32743]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T13:09:01.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.481+0000 7fb74d16f700 1 -- 172.21.15.72:0/2473942945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7481019c0 msgr2=0x7fb748101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:01.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.481+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2473942945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7481019c0 0x7fb748101de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb7380099c0 tx=0x7fb738009cd0 comp rx=0 tx=0).stop 2024-04-25T13:09:01.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.481+0000 7fb74d16f700 1 -- 172.21.15.72:0/2473942945 shutdown_connections 2024-04-25T13:09:01.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.481+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2473942945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb748103560 0x7fb748109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:01.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.481+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2473942945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7481019c0 0x7fb748101de0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:01.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.481+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2473942945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb748102bc0 0x7fb748103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:01.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.481+0000 7fb74d16f700 1 -- 172.21.15.72:0/2473942945 >> 172.21.15.72:0/2473942945 conn(0x7fb7480fcfa0 msgr2=0x7fb7480ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:01.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.481+0000 7fb74d16f700 1 -- 172.21.15.72:0/2473942945 shutdown_connections 2024-04-25T13:09:01.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.481+0000 7fb74d16f700 1 -- 172.21.15.72:0/2473942945 wait complete. 2024-04-25T13:09:01.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.482+0000 7fb74d16f700 1 Processor -- start 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.482+0000 7fb74d16f700 1 -- start start 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74d16f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7481019c0 0x7fb748198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74d16f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb748102bc0 0x7fb748199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74d16f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb748103560 0x7fb7481a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74d16f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb74810d430 con 0x7fb748102bc0 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74d16f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb74810d2b0 con 0x7fb7481019c0 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74d16f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb74810d5b0 con 0x7fb748103560 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb746d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7481019c0 0x7fb748198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74759e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb748103560 0x7fb7481a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:01.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb746d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7481019c0 0x7fb748198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:58038/0 (socket says 172.21.15.72:58038) 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb746d9d700 1 -- 172.21.15.72:0/2700335335 learned_addr learned my addr 172.21.15.72:0/2700335335 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74659c700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb748102bc0 0x7fb748199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb746d9d700 1 -- 172.21.15.72:0/2700335335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb748103560 msgr2=0x7fb7481a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb746d9d700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb748103560 0x7fb7481a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb746d9d700 1 -- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb748102bc0 msgr2=0x7fb748199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb746d9d700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb748102bc0 0x7fb748199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb746d9d700 1 -- 172.21.15.72:0/2700335335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7481a0860 con 0x7fb7481019c0 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74659c700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb748102bc0 0x7fb748199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb74759e700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb748103560 0x7fb7481a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.483+0000 7fb746d9d700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7481019c0 0x7fb748198c10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb7380038c0 tx=0x7fb7380047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.484+0000 7fb72ffff700 1 -- 172.21.15.72:0/2700335335 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738027070 con 0x7fb7481019c0 2024-04-25T13:09:01.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.484+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7481a0af0 con 0x7fb7481019c0 2024-04-25T13:09:01.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.484+0000 7fb72ffff700 1 -- 172.21.15.72:0/2700335335 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7380040e0 con 0x7fb7481019c0 2024-04-25T13:09:01.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.484+0000 7fb72ffff700 1 -- 172.21.15.72:0/2700335335 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738022df0 con 0x7fb7481019c0 2024-04-25T13:09:01.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.484+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7481a1020 con 0x7fb7481019c0 2024-04-25T13:09:01.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.484+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb748069730 con 0x7fb7481019c0 2024-04-25T13:09:01.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.486+0000 7fb72ffff700 1 -- 172.21.15.72:0/2700335335 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb73802a070 con 0x7fb7481019c0 2024-04-25T13:09:01.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.486+0000 7fb72ffff700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb7300a3c40 0x7fb7300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:01.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.487+0000 7fb74659c700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb7300a3c40 0x7fb7300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:01.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.487+0000 7fb72ffff700 1 -- 172.21.15.72:0/2700335335 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7fb7380d0640 con 0x7fb7481019c0 2024-04-25T13:09:01.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.487+0000 7fb74659c700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb7300a3c40 0x7fb7300a6100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb73c005950 tx=0x7fb73c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:01.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.491+0000 7fb72ffff700 1 -- 172.21.15.72:0/2700335335 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb738013070 con 0x7fb7481019c0 2024-04-25T13:09:01.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:01 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi193 2024-04-25T13:09:01.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:01 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi193 to 5696M 2024-04-25T13:09:01.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:01 smithi164 ceph-mon[32418]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T13:09:01.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:01.699+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb74806b8c0 con 0x7fb7300a3c40 2024-04-25T13:09:02.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:02 smithi193 ceph-mon[32743]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:02.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:02 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:09:02.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:02 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:09:02.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:02 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:02.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:02 smithi193 ceph-mon[32743]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:09:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:02 smithi072 ceph-mon[29199]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:02 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:09:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:02 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:09:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:02 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:02 smithi072 ceph-mon[29199]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:09:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:02 smithi164 ceph-mon[32418]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:02 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:09:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:02 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:09:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:02 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:02 smithi164 ceph-mon[32418]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:09:03.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:03 smithi072 ceph-mon[29199]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:09:03.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:03 smithi193 ceph-mon[32743]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:09:03.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:03 smithi164 ceph-mon[32418]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:09:04.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:04 smithi193 ceph-mon[32743]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:04.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:04 smithi193 ceph-mon[32743]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:09:04.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:04 smithi072 ceph-mon[29199]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:04.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:04 smithi072 ceph-mon[29199]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:09:04.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:04 smithi164 ceph-mon[32418]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:04.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:04 smithi164 ceph-mon[32418]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:09:05.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:05 smithi193 ceph-mon[32743]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T13:09:05.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:05 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2678866567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10681718-08e5-4452-845f-75a2299d281f"}]: dispatch 2024-04-25T13:09:05.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:05 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2678866567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10681718-08e5-4452-845f-75a2299d281f"}]': finished 2024-04-25T13:09:05.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:05 smithi193 ceph-mon[32743]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T13:09:05.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:05.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:05 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2428233175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:09:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:05 smithi072 ceph-mon[29199]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T13:09:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:05 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2678866567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10681718-08e5-4452-845f-75a2299d281f"}]: dispatch 2024-04-25T13:09:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:05 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2678866567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10681718-08e5-4452-845f-75a2299d281f"}]': finished 2024-04-25T13:09:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:05 smithi072 ceph-mon[29199]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T13:09:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:05 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2428233175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:09:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:05 smithi164 ceph-mon[32418]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T13:09:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:05 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2678866567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10681718-08e5-4452-845f-75a2299d281f"}]: dispatch 2024-04-25T13:09:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:05 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2678866567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10681718-08e5-4452-845f-75a2299d281f"}]': finished 2024-04-25T13:09:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:05 smithi164 ceph-mon[32418]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T13:09:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:05 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2428233175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:09:06.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:06 smithi193 ceph-mon[32743]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-25T13:09:06.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:06 smithi072 ceph-mon[29199]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-25T13:09:06.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:06 smithi164 ceph-mon[32418]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-25T13:09:08.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:08 smithi193 ceph-mon[32743]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T13:09:08.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:08 smithi072 ceph-mon[29199]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T13:09:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:08 smithi164 ceph-mon[32418]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T13:09:10.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:10 smithi193 ceph-mon[32743]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:09:10.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:10 smithi072 ceph-mon[29199]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:09:10.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:10 smithi164 ceph-mon[32418]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:09:12.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:12 smithi193 ceph-mon[32743]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T13:09:12.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:12 smithi072 ceph-mon[29199]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T13:09:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:12 smithi164 ceph-mon[32418]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T13:09:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:13 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:09:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:13 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:13.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:13 smithi072 ceph-mon[29199]: Deploying daemon osd.9 on smithi072 2024-04-25T13:09:13.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:13 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:09:13.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:13 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:13.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:13 smithi193 ceph-mon[32743]: Deploying daemon osd.9 on smithi072 2024-04-25T13:09:13.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:13 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:09:13.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:13 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:13.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:13 smithi164 ceph-mon[32418]: Deploying daemon osd.9 on smithi072 2024-04-25T13:09:14.227 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:14 smithi072 ceph-mon[29199]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T13:09:14.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:14 smithi193 ceph-mon[32743]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T13:09:14.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:14 smithi164 ceph-mon[32418]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T13:09:16.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:16 smithi193 ceph-mon[32743]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T13:09:16.503 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:16 smithi072 ceph-mon[29199]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T13:09:16.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:16 smithi164 ceph-mon[32418]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T13:09:18.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:18 smithi193 ceph-mon[32743]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:18.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:18 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:18.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:18 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:18.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:18 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:18.484 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:18 smithi072 ceph-mon[29199]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:18.485 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:18 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:18.485 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:18 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:18.485 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:18 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:18.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:18 smithi164 ceph-mon[32418]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:18.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:18 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:18.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:18 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:18.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:18 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:20.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:20 smithi193 ceph-mon[32743]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:20.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:20 smithi072 ceph-mon[29199]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:20.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:20 smithi164 ceph-mon[32418]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:21.406 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 9 on host 'smithi072' 2024-04-25T13:09:21.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.406+0000 7fb72ffff700 1 -- 172.21.15.72:0/2700335335 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb74806b8c0 con 0x7fb7300a3c40 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb7300a3c40 msgr2=0x7fb7300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb7300a3c40 0x7fb7300a6100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb73c005950 tx=0x7fb73c00a300 comp rx=0 tx=0).stop 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7481019c0 msgr2=0x7fb748198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7481019c0 0x7fb748198c10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb7380038c0 tx=0x7fb7380047b0 comp rx=0 tx=0).stop 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 shutdown_connections 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb748103560 0x7fb7481a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7481019c0 0x7fb748198c10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb7300a3c40 0x7fb7300a6100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 --2- 172.21.15.72:0/2700335335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb748102bc0 0x7fb748199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:21.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 >> 172.21.15.72:0/2700335335 conn(0x7fb7480fcfa0 msgr2=0x7fb74806b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:21.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 shutdown_connections 2024-04-25T13:09:21.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:09:21.408+0000 7fb74d16f700 1 -- 172.21.15.72:0/2700335335 wait complete. 2024-04-25T13:09:22.426 DEBUG:teuthology.orchestra.run.smithi072:osd.9> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T13:09:22.429 INFO:tasks.cephadm:Deploying osd.10 on smithi164 with /dev/vg_nvme/lv_1... 2024-04-25T13:09:22.429 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:09:22.674 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:09:22.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:22 smithi193 ceph-mon[32743]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:22.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:22 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:22.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:22 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:22.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:22 smithi193 ceph-mon[32743]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:09:22.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:22 smithi072 ceph-mon[29199]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:22.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:22 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:22.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:22 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:22.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:22 smithi072 ceph-mon[29199]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:09:22.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:22 smithi164 ceph-mon[32418]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:22.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:22 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:22.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:22 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:22.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:22 smithi164 ceph-mon[32418]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:09:23.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:23 smithi072 ceph-mon[29199]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:09:23.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:23 smithi072 ceph-mon[29199]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T13:09:23.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:23 smithi072 ceph-mon[29199]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:09:23.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:23.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:23.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:23 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:23.525 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:09:23 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:09:23.375+0000 7f342f8db700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T13:09:23.525 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:09:23 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:09:23.389+0000 7f342b6ce700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:09:23.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:23 smithi164 ceph-mon[32418]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:09:23.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:23 smithi164 ceph-mon[32418]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T13:09:23.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:23 smithi164 ceph-mon[32418]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:09:23.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:23.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:23.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:23 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:23 smithi193 ceph-mon[32743]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:09:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:23 smithi193 ceph-mon[32743]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T13:09:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:23 smithi193 ceph-mon[32743]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-25T13:09:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:23 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:24.506 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:09:24.506 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:09:24.506 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-25T13:09:24.506 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-25T13:09:24.506 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126712 s, 828 MB/s 2024-04-25T13:09:24.506 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-25T13:09:24.598 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:24 smithi072 ceph-mon[29199]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:24.598 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:24 smithi072 ceph-mon[29199]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:09:24.598 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:24 smithi072 ceph-mon[29199]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T13:09:24.598 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:24 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:24.598 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:24 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:24.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:24 smithi193 ceph-mon[32743]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:24.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:24 smithi193 ceph-mon[32743]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:09:24.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:24 smithi193 ceph-mon[32743]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T13:09:24.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:24 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:24.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:24 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:24.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:24 smithi164 ceph-mon[32418]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:24.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:24 smithi164 ceph-mon[32418]: from='osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-25T13:09:24.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:24 smithi164 ceph-mon[32418]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T13:09:24.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:24 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:24.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:24 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:25.130 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2024-04-25T13:09:25.375 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:09:25.641 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:25 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:09:25.641 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:25 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:09:25.641 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:25 smithi164 ceph-mon[32418]: osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] boot 2024-04-25T13:09:25.641 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:25 smithi164 ceph-mon[32418]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T13:09:25.641 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:25 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:25.642 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:25 smithi164 ceph-mon[32418]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:25.674 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:25 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:09:25.674 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:25 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:09:25.674 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:25 smithi072 ceph-mon[29199]: osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] boot 2024-04-25T13:09:25.674 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:25 smithi072 ceph-mon[29199]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T13:09:25.674 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:25 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:25.674 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:25 smithi072 ceph-mon[29199]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:25 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:09:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:25 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:09:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:25 smithi193 ceph-mon[32743]: osd.9 [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] boot 2024-04-25T13:09:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:25 smithi193 ceph-mon[32743]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T13:09:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:25 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:09:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:25 smithi193 ceph-mon[32743]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi072 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi072 to 4272M 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:26.487 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.488 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:26.488 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:26.488 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:26.488 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:26 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi072 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi072 to 4272M 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:26.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:26.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:26.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:26 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T13:09:26.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi072 2024-04-25T13:09:26.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:26.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi072 to 4272M 2024-04-25T13:09:26.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:26.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:26.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:26.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:26.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:26.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:26 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:26.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.968+0000 7f1247afb700 1 -- 172.21.15.164:0/737303868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240100db0 msgr2=0x7f12401011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:26.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.968+0000 7f1247afb700 1 --2- 172.21.15.164:0/737303868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240100db0 0x7f12401011d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f123c0099c0 tx=0x7f123c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:09:26.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.968+0000 7f1247afb700 1 -- 172.21.15.164:0/737303868 shutdown_connections 2024-04-25T13:09:26.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.968+0000 7f1247afb700 1 --2- 172.21.15.164:0/737303868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12401047d0 0x7f124010b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:26.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.968+0000 7f1247afb700 1 --2- 172.21.15.164:0/737303868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240100db0 0x7f12401011d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:26.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.968+0000 7f1247afb700 1 --2- 172.21.15.164:0/737303868 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1240103e30 0x7f1240104290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:26.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.968+0000 7f1247afb700 1 -- 172.21.15.164:0/737303868 >> 172.21.15.164:0/737303868 conn(0x7f12400fc6f0 msgr2=0x7f12400feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:26.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.968+0000 7f1247afb700 1 -- 172.21.15.164:0/737303868 shutdown_connections 2024-04-25T13:09:26.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.968+0000 7f1247afb700 1 -- 172.21.15.164:0/737303868 wait complete. 2024-04-25T13:09:26.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1247afb700 1 Processor -- start 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1247afb700 1 -- start start 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1247afb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1240100db0 0x7f1240198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1247afb700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240103e30 0x7f12401990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1247afb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12401047d0 0x7f12401a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1247afb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f124010e560 con 0x7f12401047d0 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1247afb700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f124010e3e0 con 0x7f1240103e30 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1247afb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f124010e6e0 con 0x7f1240100db0 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1245096700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240103e30 0x7f12401990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1245096700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240103e30 0x7f12401990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56972/0 (socket says 172.21.15.164:56972) 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1245096700 1 -- 172.21.15.164:0/385969442 learned_addr learned my addr 172.21.15.164:0/385969442 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1245897700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1240100db0 0x7f1240198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1246098700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12401047d0 0x7f12401a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1245096700 1 -- 172.21.15.164:0/385969442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1240100db0 msgr2=0x7f1240198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1245096700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1240100db0 0x7f1240198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1245096700 1 -- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12401047d0 msgr2=0x7f12401a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1245096700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12401047d0 0x7f12401a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1245096700 1 -- 172.21.15.164:0/385969442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12401a07d0 con 0x7f1240103e30 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.969+0000 7f1246098700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12401047d0 0x7f12401a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.970+0000 7f1245096700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240103e30 0x7f12401990b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f123000c9b0 tx=0x7f123000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.970+0000 7f1236ffd700 1 -- 172.21.15.164:0/385969442 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230007b00 con 0x7f1240103e30 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.970+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12401a0ac0 con 0x7f1240103e30 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.970+0000 7f1236ffd700 1 -- 172.21.15.164:0/385969442 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1230007ca0 con 0x7f1240103e30 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.970+0000 7f1236ffd700 1 -- 172.21.15.164:0/385969442 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230018660 con 0x7f1240103e30 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.970+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12401a1000 con 0x7f1240103e30 2024-04-25T13:09:26.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.971+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1240069730 con 0x7f1240103e30 2024-04-25T13:09:26.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.972+0000 7f1236ffd700 1 -- 172.21.15.164:0/385969442 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f123000f450 con 0x7f1240103e30 2024-04-25T13:09:26.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.972+0000 7f1236ffd700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f122c0a3c40 0x7f122c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:26.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.972+0000 7f1236ffd700 1 -- 172.21.15.164:0/385969442 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7f12300cfe30 con 0x7f1240103e30 2024-04-25T13:09:26.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.973+0000 7f1245897700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f122c0a3c40 0x7f122c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:26.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.973+0000 7f1245897700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f122c0a3c40 0x7f122c0a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f123c006010 tx=0x7f123c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:26.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:26.976+0000 7f1236ffd700 1 -- 172.21.15.164:0/385969442 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1230014030 con 0x7f1240103e30 2024-04-25T13:09:27.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:27.186+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1240062e80 con 0x7f122c0a3c40 2024-04-25T13:09:27.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:27 smithi164 ceph-mon[32418]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:27.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:27 smithi164 ceph-mon[32418]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:09:27.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:09:27.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:09:27.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:27 smithi193 ceph-mon[32743]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:27 smithi193 ceph-mon[32743]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:09:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:09:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:09:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:27.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:27 smithi072 ceph-mon[29199]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:27.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:27 smithi072 ceph-mon[29199]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:09:27.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:09:27.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:09:27.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:30.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:30 smithi193 ceph-mon[32743]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:30.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:30 smithi193 ceph-mon[32743]: from='client.? 172.21.15.164:0/2636959008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcbcf574-0a98-411a-ac22-fe6520a5c1eb"}]: dispatch 2024-04-25T13:09:30.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:30 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcbcf574-0a98-411a-ac22-fe6520a5c1eb"}]: dispatch 2024-04-25T13:09:30.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:30 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcbcf574-0a98-411a-ac22-fe6520a5c1eb"}]': finished 2024-04-25T13:09:30.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:30 smithi193 ceph-mon[32743]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T13:09:30.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:30 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:30 smithi072 ceph-mon[29199]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:30 smithi072 ceph-mon[29199]: from='client.? 172.21.15.164:0/2636959008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcbcf574-0a98-411a-ac22-fe6520a5c1eb"}]: dispatch 2024-04-25T13:09:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:30 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcbcf574-0a98-411a-ac22-fe6520a5c1eb"}]: dispatch 2024-04-25T13:09:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:30 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcbcf574-0a98-411a-ac22-fe6520a5c1eb"}]': finished 2024-04-25T13:09:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:30 smithi072 ceph-mon[29199]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T13:09:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:30 smithi164 ceph-mon[32418]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:30 smithi164 ceph-mon[32418]: from='client.? 172.21.15.164:0/2636959008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcbcf574-0a98-411a-ac22-fe6520a5c1eb"}]: dispatch 2024-04-25T13:09:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:30 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcbcf574-0a98-411a-ac22-fe6520a5c1eb"}]: dispatch 2024-04-25T13:09:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:30 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcbcf574-0a98-411a-ac22-fe6520a5c1eb"}]': finished 2024-04-25T13:09:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:30 smithi164 ceph-mon[32418]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T13:09:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:30 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:31.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:31 smithi193 ceph-mon[32743]: from='client.? 172.21.15.164:0/2840719713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:09:31.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:31 smithi072 ceph-mon[29199]: from='client.? 172.21.15.164:0/2840719713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:09:31.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:31 smithi164 ceph-mon[32418]: from='client.? 172.21.15.164:0/2840719713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:09:32.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:32 smithi193 ceph-mon[32743]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:32.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:32 smithi072 ceph-mon[29199]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:32 smithi164 ceph-mon[32418]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:34.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:34 smithi164 ceph-mon[32418]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:34.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:34 smithi193 ceph-mon[32743]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:34.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:34 smithi072 ceph-mon[29199]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:36.428 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:36 smithi164 ceph-mon[32418]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:36.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:36 smithi193 ceph-mon[32743]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:36.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:36 smithi072 ceph-mon[29199]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:38.244 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:38 smithi164 ceph-mon[32418]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:38.244 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:09:38.245 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:38.245 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:38 smithi164 ceph-mon[32418]: Deploying daemon osd.10 on smithi164 2024-04-25T13:09:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:38 smithi193 ceph-mon[32743]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:09:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:38 smithi193 ceph-mon[32743]: Deploying daemon osd.10 on smithi164 2024-04-25T13:09:38.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:38 smithi072 ceph-mon[29199]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:38.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:09:38.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:38.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:38 smithi072 ceph-mon[29199]: Deploying daemon osd.10 on smithi164 2024-04-25T13:09:40.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:40 smithi193 ceph-mon[32743]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:40.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:40 smithi072 ceph-mon[29199]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:40.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:40 smithi164 ceph-mon[32418]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:42.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:42 smithi193 ceph-mon[32743]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:42.498 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:42 smithi164 ceph-mon[32418]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:42.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:42 smithi072 ceph-mon[29199]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:43.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:43.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:43.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:43.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:43 smithi193 ceph-mon[32743]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:44.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:44.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:44.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:44.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:43 smithi072 ceph-mon[29199]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:44.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:44.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:44.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:44.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:43 smithi164 ceph-mon[32418]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:46.423 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:46 smithi164 ceph-mon[32418]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:46.423 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:46.423 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:46.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:46.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:46.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:46.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:46 smithi193 ceph-mon[32743]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:46.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:46.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:46.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:46.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:46.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:46.505 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 10 on host 'smithi164' 2024-04-25T13:09:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.504+0000 7f1236ffd700 1 -- 172.21.15.164:0/385969442 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1240062e80 con 0x7f122c0a3c40 2024-04-25T13:09:46.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f122c0a3c40 msgr2=0x7f122c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:46.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f122c0a3c40 0x7f122c0a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f123c006010 tx=0x7f123c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:09:46.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240103e30 msgr2=0x7f12401990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:46.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240103e30 0x7f12401990b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f123000c9b0 tx=0x7f123000ccc0 comp rx=0 tx=0).stop 2024-04-25T13:09:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 shutdown_connections 2024-04-25T13:09:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f122c0a3c40 0x7f122c0a6100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12401047d0 0x7f12401a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1240100db0 0x7f1240198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 --2- 172.21.15.164:0/385969442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1240103e30 0x7f12401990b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 >> 172.21.15.164:0/385969442 conn(0x7f12400fc6f0 msgr2=0x7f1240107130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 shutdown_connections 2024-04-25T13:09:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T13:09:46.507+0000 7f1247afb700 1 -- 172.21.15.164:0/385969442 wait complete. 2024-04-25T13:09:46.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:46 smithi072 ceph-mon[29199]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:46.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:46.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:46.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:46.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:46.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:47.177 DEBUG:teuthology.orchestra.run.smithi164:osd.10> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T13:09:47.180 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2024-04-25T13:09:47.180 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:09:47.440 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:09:47.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:47 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:47.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:47 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:47.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:47 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:47.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:47 smithi193 ceph-mon[32743]: from='osd.10 [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:09:47.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:47 smithi193 ceph-mon[32743]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:09:47.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:47 smithi193 ceph-mon[32743]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:47.764 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:47 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:47.764 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:47 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:47.764 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:47 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:47.764 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:47 smithi164 ceph-mon[32418]: from='osd.10 [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:09:47.764 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:47 smithi164 ceph-mon[32418]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:09:47.764 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:47 smithi164 ceph-mon[32418]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:47.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:47.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:47.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:47 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:47.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:47 smithi072 ceph-mon[29199]: from='osd.10 [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:09:47.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:47 smithi072 ceph-mon[29199]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:09:47.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:47 smithi072 ceph-mon[29199]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:48.647 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:48 smithi164 ceph-mon[32418]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:09:48.647 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:48 smithi164 ceph-mon[32418]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T13:09:48.647 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:48.647 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:48 smithi164 ceph-mon[32418]: from='osd.10 [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:09:48.648 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:48 smithi164 ceph-mon[32418]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:09:48.724 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:48 smithi193 ceph-mon[32743]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:09:48.724 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:48 smithi193 ceph-mon[32743]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T13:09:48.724 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:48.724 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:48 smithi193 ceph-mon[32743]: from='osd.10 [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:09:48.724 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:48 smithi193 ceph-mon[32743]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:09:48.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:48 smithi072 ceph-mon[29199]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:09:48.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:48 smithi072 ceph-mon[29199]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T13:09:48.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:48.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:48 smithi072 ceph-mon[29199]: from='osd.10 [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:09:48.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:48 smithi072 ceph-mon[29199]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T13:09:49.390 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:09:49.390 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:09:49.390 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-04-25T13:09:49.390 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-04-25T13:09:49.390 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112976 s, 928 MB/s 2024-04-25T13:09:49.390 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-04-25T13:09:49.721 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:49 smithi193 ceph-mon[32743]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:09:49.721 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:49 smithi193 ceph-mon[32743]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T13:09:49.721 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:49 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:49.721 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:49 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:49.721 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:49 smithi193 ceph-mon[32743]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:49.721 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:49 smithi193 ceph-mon[32743]: from='osd.10 ' entity='osd.10' 2024-04-25T13:09:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:49 smithi072 ceph-mon[29199]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:09:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:49 smithi072 ceph-mon[29199]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T13:09:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:49 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:49 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:49 smithi072 ceph-mon[29199]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:49 smithi072 ceph-mon[29199]: from='osd.10 ' entity='osd.10' 2024-04-25T13:09:49.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:49 smithi164 ceph-mon[32418]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T13:09:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:49 smithi164 ceph-mon[32418]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T13:09:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:49 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:49 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:49 smithi164 ceph-mon[32418]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:49 smithi164 ceph-mon[32418]: from='osd.10 ' entity='osd.10' 2024-04-25T13:09:49.808 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:09:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:09:49.398+0000 7fef939c3700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T13:09:49.808 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:09:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:09:49.564+0000 7fef8b7ae700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:09:50.101 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-04-25T13:09:50.340 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:09:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:09:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:09:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T13:09:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi164 2024-04-25T13:09:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi164 to 4272M 2024-04-25T13:09:50.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:50.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:50.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:50 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi164 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi164 to 4272M 2024-04-25T13:09:50.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:50.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:50.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:50 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:50.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:09:50.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:09:50.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:50.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi164 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi164 to 4272M 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:50.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:09:50.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:50 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:51.748 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:51 smithi193 ceph-mon[32743]: osd.10 [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] boot 2024-04-25T13:09:51.749 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:51 smithi193 ceph-mon[32743]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T13:09:51.749 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:51.749 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:51 smithi193 ceph-mon[32743]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:51.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:51 smithi072 ceph-mon[29199]: osd.10 [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] boot 2024-04-25T13:09:51.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:51 smithi072 ceph-mon[29199]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T13:09:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:51 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:51 smithi072 ceph-mon[29199]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:51.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:51 smithi164 ceph-mon[32418]: osd.10 [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] boot 2024-04-25T13:09:51.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:51 smithi164 ceph-mon[32418]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T13:09:51.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:51 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:09:51.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:51 smithi164 ceph-mon[32418]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:09:51.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.960+0000 7fcd66960700 1 -- 172.21.15.193:0/2538883345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60102bc0 msgr2=0x7fcd60103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:51.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.960+0000 7fcd66960700 1 --2- 172.21.15.193:0/2538883345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60102bc0 0x7fcd60103020 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009990 tx=0x7fcd54009ca0 comp rx=0 tx=0).stop 2024-04-25T13:09:51.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.960+0000 7fcd66960700 1 -- 172.21.15.193:0/2538883345 shutdown_connections 2024-04-25T13:09:51.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.960+0000 7fcd66960700 1 --2- 172.21.15.193:0/2538883345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd60103560 0x7fcd60109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:51.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.960+0000 7fcd66960700 1 --2- 172.21.15.193:0/2538883345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd601019c0 0x7fcd60101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.960+0000 7fcd66960700 1 --2- 172.21.15.193:0/2538883345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60102bc0 0x7fcd60103020 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.960+0000 7fcd66960700 1 -- 172.21.15.193:0/2538883345 >> 172.21.15.193:0/2538883345 conn(0x7fcd600fcfa0 msgr2=0x7fcd600ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.960+0000 7fcd66960700 1 -- 172.21.15.193:0/2538883345 shutdown_connections 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.960+0000 7fcd66960700 1 -- 172.21.15.193:0/2538883345 wait complete. 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd66960700 1 Processor -- start 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd66960700 1 -- start start 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd66960700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd601019c0 0x7fcd60197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd66960700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd60102bc0 0x7fcd601980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd66960700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60103560 0x7fcd6019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd66960700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010d520 con 0x7fcd601019c0 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd66960700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010d3a0 con 0x7fcd60102bc0 2024-04-25T13:09:51.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd66960700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010d6a0 con 0x7fcd60103560 2024-04-25T13:09:51.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd64efd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60103560 0x7fcd6019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:51.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd64efd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60103560 0x7fcd6019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:52574/0 (socket says 172.21.15.193:52574) 2024-04-25T13:09:51.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.961+0000 7fcd64efd700 1 -- 172.21.15.193:0/817540089 learned_addr learned my addr 172.21.15.193:0/817540089 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:09:51.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd5ffff700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd601019c0 0x7fcd60197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:51.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd5f7fe700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd60102bc0 0x7fcd601980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd64efd700 1 -- 172.21.15.193:0/817540089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd60102bc0 msgr2=0x7fcd601980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd64efd700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd60102bc0 0x7fcd601980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd64efd700 1 -- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd601019c0 msgr2=0x7fcd60197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd64efd700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd601019c0 0x7fcd60197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd64efd700 1 -- 172.21.15.193:0/817540089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6019f7d0 con 0x7fcd60103560 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd5f7fe700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd60102bc0 0x7fcd601980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd5ffff700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd601019c0 0x7fcd60197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd64efd700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60103560 0x7fcd6019f130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800b9a0 tx=0x7fcd5800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd5d7fa700 1 -- 172.21.15.193:0/817540089 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58019070 con 0x7fcd60103560 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6019fac0 con 0x7fcd60103560 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd5d7fa700 1 -- 172.21.15.193:0/817540089 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd58014440 con 0x7fcd60103560 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd5d7fa700 1 -- 172.21.15.193:0/817540089 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800ce60 con 0x7fcd60103560 2024-04-25T13:09:51.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.962+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6019ffb0 con 0x7fcd60103560 2024-04-25T13:09:51.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.963+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd60069730 con 0x7fcd60103560 2024-04-25T13:09:51.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.964+0000 7fcd5d7fa700 1 -- 172.21.15.193:0/817540089 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcd5801c070 con 0x7fcd60103560 2024-04-25T13:09:51.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.965+0000 7fcd5d7fa700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcd480a3c40 0x7fcd480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:51.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.965+0000 7fcd5d7fa700 1 -- 172.21.15.193:0/817540089 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7fcd580cf210 con 0x7fcd60103560 2024-04-25T13:09:51.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.965+0000 7fcd5ffff700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcd480a3c40 0x7fcd480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:51.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.966+0000 7fcd5ffff700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcd480a3c40 0x7fcd480a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005950 tx=0x7fcd500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:51.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:51.969+0000 7fcd5d7fa700 1 -- 172.21.15.193:0/817540089 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd5809c2b0 con 0x7fcd60103560 2024-04-25T13:09:52.176 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:09:52.176+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcd60062e80 con 0x7fcd480a3c40 2024-04-25T13:09:52.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:52 smithi072 ceph-mon[29199]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:09:52.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:52 smithi072 ceph-mon[29199]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:09:52.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:52 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:09:52.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:52 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:09:52.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:52 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:52.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:52 smithi193 ceph-mon[32743]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:09:52.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:52 smithi193 ceph-mon[32743]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:09:52.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:52 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:09:52.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:52 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:09:52.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:52 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:52.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:52 smithi164 ceph-mon[32418]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:09:52.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:52 smithi164 ceph-mon[32418]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:09:52.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:52 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:09:52.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:52 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:09:52.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:52 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:53.763 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:53 smithi193 ceph-mon[32743]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:09:53.763 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:53 smithi193 ceph-mon[32743]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:09:53.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:53 smithi072 ceph-mon[29199]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:09:53.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:53 smithi072 ceph-mon[29199]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:09:53.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:53 smithi164 ceph-mon[32418]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:09:53.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:53 smithi164 ceph-mon[32418]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:09:54.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:54 smithi072 ceph-mon[29199]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:09:54.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:54 smithi164 ceph-mon[32418]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:09:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:54 smithi193 ceph-mon[32743]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:09:55.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:55 smithi072 ceph-mon[29199]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T13:09:55.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:55 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/1994059862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36fabfaa-4071-45f7-8d95-1a989dcba56b"}]: dispatch 2024-04-25T13:09:55.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:55 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36fabfaa-4071-45f7-8d95-1a989dcba56b"}]: dispatch 2024-04-25T13:09:55.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:55 smithi072 ceph-mon[29199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36fabfaa-4071-45f7-8d95-1a989dcba56b"}]': finished 2024-04-25T13:09:55.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:55 smithi072 ceph-mon[29199]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:09:55.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:55 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:09:55.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:55 smithi072 ceph-mon[29199]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 159 KiB/s, 0 objects/s recovering 2024-04-25T13:09:55.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:55 smithi072 ceph-mon[29199]: from='client.? 172.21.15.193:0/15805927' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:09:55.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:55 smithi164 ceph-mon[32418]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T13:09:55.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:55 smithi164 ceph-mon[32418]: from='client.? 172.21.15.193:0/1994059862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36fabfaa-4071-45f7-8d95-1a989dcba56b"}]: dispatch 2024-04-25T13:09:55.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:55 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36fabfaa-4071-45f7-8d95-1a989dcba56b"}]: dispatch 2024-04-25T13:09:55.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:55 smithi164 ceph-mon[32418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36fabfaa-4071-45f7-8d95-1a989dcba56b"}]': finished 2024-04-25T13:09:55.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:55 smithi164 ceph-mon[32418]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:09:55.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:55 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:09:55.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:55 smithi164 ceph-mon[32418]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 159 KiB/s, 0 objects/s recovering 2024-04-25T13:09:55.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:55 smithi164 ceph-mon[32418]: from='client.? 172.21.15.193:0/15805927' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:09:55.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:55 smithi193 ceph-mon[32743]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T13:09:55.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:55 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/1994059862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36fabfaa-4071-45f7-8d95-1a989dcba56b"}]: dispatch 2024-04-25T13:09:55.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:55 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36fabfaa-4071-45f7-8d95-1a989dcba56b"}]: dispatch 2024-04-25T13:09:55.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:55 smithi193 ceph-mon[32743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36fabfaa-4071-45f7-8d95-1a989dcba56b"}]': finished 2024-04-25T13:09:55.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:55 smithi193 ceph-mon[32743]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:09:55.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:55 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:09:55.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:55 smithi193 ceph-mon[32743]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 159 KiB/s, 0 objects/s recovering 2024-04-25T13:09:55.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:55 smithi193 ceph-mon[32743]: from='client.? 172.21.15.193:0/15805927' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:09:58.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:09:58 smithi193 ceph-mon[32743]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T13:09:58.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:09:58 smithi072 ceph-mon[29199]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T13:09:58.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:09:58 smithi164 ceph-mon[32418]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T13:10:00.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:00 smithi072 ceph-mon[29199]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:10:00.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:00 smithi072 ceph-mon[29199]: overall HEALTH_OK 2024-04-25T13:10:00.400 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:00 smithi193 ceph-mon[32743]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:10:00.400 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:00 smithi193 ceph-mon[32743]: overall HEALTH_OK 2024-04-25T13:10:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:00 smithi164 ceph-mon[32418]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:10:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:00 smithi164 ceph-mon[32418]: overall HEALTH_OK 2024-04-25T13:10:02.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:02 smithi193 ceph-mon[32743]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T13:10:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:02 smithi072 ceph-mon[29199]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T13:10:02.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:02 smithi164 ceph-mon[32418]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T13:10:03.357 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:10:03.357 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:03.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:10:03.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:03.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:10:03.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:04.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:04 smithi193 ceph-mon[32743]: Deploying daemon osd.11 on smithi193 2024-04-25T13:10:04.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:04 smithi193 ceph-mon[32743]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T13:10:04.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:04 smithi072 ceph-mon[29199]: Deploying daemon osd.11 on smithi193 2024-04-25T13:10:04.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:04 smithi072 ceph-mon[29199]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T13:10:04.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:04 smithi164 ceph-mon[32418]: Deploying daemon osd.11 on smithi193 2024-04-25T13:10:04.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:04 smithi164 ceph-mon[32418]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T13:10:06.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:06 smithi193 ceph-mon[32743]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:06.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:06 smithi072 ceph-mon[29199]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:06.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:06 smithi164 ceph-mon[32418]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:08.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:08 smithi193 ceph-mon[32743]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:08.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:08 smithi072 ceph-mon[29199]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:08.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:08.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:08.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:08 smithi164 ceph-mon[32418]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:10.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:10 smithi193 ceph-mon[32743]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:10.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:10 smithi072 ceph-mon[29199]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:10.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:10 smithi164 ceph-mon[32418]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:11.491 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.491+0000 7fcd5d7fa700 1 -- 172.21.15.193:0/817540089 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcd60062e80 con 0x7fcd480a3c40 2024-04-25T13:10:11.492 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2024-04-25T13:10:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.493+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcd480a3c40 msgr2=0x7fcd480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.493+0000 7fcd66960700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcd480a3c40 0x7fcd480a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005950 tx=0x7fcd500058e0 comp rx=0 tx=0).stop 2024-04-25T13:10:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.493+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60103560 msgr2=0x7fcd6019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.493+0000 7fcd66960700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60103560 0x7fcd6019f130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800b9a0 tx=0x7fcd5800bd60 comp rx=0 tx=0).stop 2024-04-25T13:10:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.493+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 shutdown_connections 2024-04-25T13:10:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.493+0000 7fcd66960700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd60103560 0x7fcd6019f130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.493+0000 7fcd66960700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcd480a3c40 0x7fcd480a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.493+0000 7fcd66960700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd601019c0 0x7fcd60197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.493+0000 7fcd66960700 1 --2- 172.21.15.193:0/817540089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd60102bc0 0x7fcd601980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.494+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 >> 172.21.15.193:0/817540089 conn(0x7fcd600fcfa0 msgr2=0x7fcd60105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.494+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 shutdown_connections 2024-04-25T13:10:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:11.494+0000 7fcd66960700 1 -- 172.21.15.193:0/817540089 wait complete. 2024-04-25T13:10:12.149 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T13:10:12.151 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T13:10:12.151 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:10:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:10:12.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:12 smithi193 ceph-mon[32743]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:12.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:12 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:12.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:12 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:12.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:12 smithi193 ceph-mon[32743]: from='osd.11 [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:10:12.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:12 smithi193 ceph-mon[32743]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:10:12.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:12 smithi072 ceph-mon[29199]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:12.521 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:12 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:12.521 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:12 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:12.521 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:12 smithi072 ceph-mon[29199]: from='osd.11 [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:10:12.521 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:12 smithi072 ceph-mon[29199]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:10:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:12 smithi164 ceph-mon[32418]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:12 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:12 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:12 smithi164 ceph-mon[32418]: from='osd.11 [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:10:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:12 smithi164 ceph-mon[32418]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:10:13.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:13 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:13.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:13 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:13.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:13 smithi193 ceph-mon[32743]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:10:13.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:13 smithi193 ceph-mon[32743]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:10:13.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:13 smithi193 ceph-mon[32743]: from='osd.11 [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:10:13.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:13 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:13.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:13 smithi193 ceph-mon[32743]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:10:13.560 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:13 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:13.561 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:13 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:13.561 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:13 smithi072 ceph-mon[29199]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:10:13.561 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:13 smithi072 ceph-mon[29199]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:10:13.561 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:13 smithi072 ceph-mon[29199]: from='osd.11 [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:10:13.561 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:13 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:13.561 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:13 smithi072 ceph-mon[29199]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:10:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:13 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:13 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:13 smithi164 ceph-mon[32418]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:10:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:13 smithi164 ceph-mon[32418]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:10:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:13 smithi164 ceph-mon[32418]: from='osd.11 [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:10:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:13 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:13.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:13 smithi164 ceph-mon[32418]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T13:10:13.907 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:10:13 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:10:13.485+0000 7faddf09b700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T13:10:13.907 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:10:13 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:10:13.506+0000 7faddae8e700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:10:14.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.055+0000 7f3d611cd700 1 -- 172.21.15.72:0/2701956651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c103ca0 msgr2=0x7f3d5c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:14.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.055+0000 7f3d611cd700 1 --2- 172.21.15.72:0/2701956651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c103ca0 0x7f3d5c104100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0099c0 tx=0x7f3d4c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.055+0000 7f3d611cd700 1 -- 172.21.15.72:0/2701956651 shutdown_connections 2024-04-25T13:10:14.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.055+0000 7f3d611cd700 1 --2- 172.21.15.72:0/2701956651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d5c104640 0x7f3d5c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.055+0000 7f3d611cd700 1 --2- 172.21.15.72:0/2701956651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c102aa0 0x7f3d5c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.055+0000 7f3d611cd700 1 --2- 172.21.15.72:0/2701956651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c103ca0 0x7f3d5c104100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.055+0000 7f3d611cd700 1 -- 172.21.15.72:0/2701956651 >> 172.21.15.72:0/2701956651 conn(0x7f3d5c0fe040 msgr2=0x7f3d5c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:14.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.055+0000 7f3d611cd700 1 -- 172.21.15.72:0/2701956651 shutdown_connections 2024-04-25T13:10:14.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.055+0000 7f3d611cd700 1 -- 172.21.15.72:0/2701956651 wait complete. 2024-04-25T13:10:14.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.056+0000 7f3d611cd700 1 Processor -- start 2024-04-25T13:10:14.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.056+0000 7f3d611cd700 1 -- start start 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.056+0000 7f3d611cd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c102aa0 0x7f3d5c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.056+0000 7f3d611cd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c103ca0 0x7f3d5c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.056+0000 7f3d611cd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d5c104640 0x7f3d5c1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.056+0000 7f3d611cd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c10e5b0 con 0x7f3d5c102aa0 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.056+0000 7f3d611cd700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c10e430 con 0x7f3d5c104640 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.056+0000 7f3d611cd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c10e730 con 0x7f3d5c103ca0 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c102aa0 0x7f3d5c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c102aa0 0x7f3d5c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34244/0 (socket says 172.21.15.72:34244) 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5ad9d700 1 -- 172.21.15.72:0/3321364130 learned_addr learned my addr 172.21.15.72:0/3321364130 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5a59c700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c103ca0 0x7f3d5c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5b59e700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d5c104640 0x7f3d5c1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:14.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5ad9d700 1 -- 172.21.15.72:0/3321364130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c103ca0 msgr2=0x7f3d5c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5ad9d700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c103ca0 0x7f3d5c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5ad9d700 1 -- 172.21.15.72:0/3321364130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d5c104640 msgr2=0x7f3d5c1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5ad9d700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d5c104640 0x7f3d5c1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5ad9d700 1 -- 172.21.15.72:0/3321364130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c1a0900 con 0x7f3d5c102aa0 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5b59e700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d5c104640 0x7f3d5c1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d5ad9d700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c102aa0 0x7f3d5c198c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000b500 tx=0x7f3d5000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d43fff700 1 -- 172.21.15.72:0/3321364130 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50015020 con 0x7f3d5c102aa0 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5c1a0bf0 con 0x7f3d5c102aa0 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d43fff700 1 -- 172.21.15.72:0/3321364130 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d50011ce0 con 0x7f3d5c102aa0 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d43fff700 1 -- 172.21.15.72:0/3321364130 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50010d20 con 0x7f3d5c102aa0 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.057+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5c1a10b0 con 0x7f3d5c102aa0 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.058+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d5c069730 con 0x7f3d5c102aa0 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.059+0000 7f3d43fff700 1 -- 172.21.15.72:0/3321364130 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3d50018020 con 0x7f3d5c102aa0 2024-04-25T13:10:14.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.060+0000 7f3d43fff700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3d440a3c40 0x7f3d440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:14.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.060+0000 7f3d5a59c700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3d440a3c40 0x7f3d440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:14.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.060+0000 7f3d43fff700 1 -- 172.21.15.72:0/3321364130 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6847+0+0 (secure 0 0 0) 0x7f3d500cfb20 con 0x7f3d5c102aa0 2024-04-25T13:10:14.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.060+0000 7f3d5a59c700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3d440a3c40 0x7f3d440a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009990 tx=0x7f3d4c00b910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:14.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.064+0000 7f3d43fff700 1 -- 172.21.15.72:0/3321364130 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d500cfe20 con 0x7f3d5c102aa0 2024-04-25T13:10:14.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.261+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3d5c02e190 con 0x7f3d5c102aa0 2024-04-25T13:10:14.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.261+0000 7f3d43fff700 1 -- 172.21.15.72:0/3321364130 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f3d5001f020 con 0x7f3d5c102aa0 2024-04-25T13:10:14.260 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:14.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3d440a3c40 msgr2=0x7f3d440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:14.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3d440a3c40 0x7f3d440a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009990 tx=0x7f3d4c00b910 comp rx=0 tx=0).stop 2024-04-25T13:10:14.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c102aa0 msgr2=0x7f3d5c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:14.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c102aa0 0x7f3d5c198c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000b500 tx=0x7f3d5000b810 comp rx=0 tx=0).stop 2024-04-25T13:10:14.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 shutdown_connections 2024-04-25T13:10:14.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d5c104640 0x7f3d5c1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3d440a3c40 0x7f3d440a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5c102aa0 0x7f3d5c198c80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 --2- 172.21.15.72:0/3321364130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c103ca0 0x7f3d5c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:14.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 >> 172.21.15.72:0/3321364130 conn(0x7f3d5c0fe040 msgr2=0x7f3d5c107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:14.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 shutdown_connections 2024-04-25T13:10:14.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:14.264+0000 7f3d611cd700 1 -- 172.21.15.72:0/3321364130 wait complete. 2024-04-25T13:10:14.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:14 smithi193 ceph-mon[32743]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:14.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:14 smithi193 ceph-mon[32743]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:10:14.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:14 smithi193 ceph-mon[32743]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T13:10:14.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:14.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:14 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:14.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:14 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3321364130' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:10:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:14 smithi072 ceph-mon[29199]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:14 smithi072 ceph-mon[29199]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:10:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:14 smithi072 ceph-mon[29199]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T13:10:14.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:14.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:14 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:14.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:14 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3321364130' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:10:14.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:14 smithi164 ceph-mon[32418]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:14.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:14 smithi164 ceph-mon[32418]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T13:10:14.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:14 smithi164 ceph-mon[32418]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T13:10:14.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:14.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:14 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:14.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:14 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3321364130' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:10:14.982 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":82,"num_osds":12,"num_up_osds":11,"osd_up_since":1714050590,"num_in_osds":12,"osd_in_since":1714050594,"num_remapped_pgs":0} 2024-04-25T13:10:15.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:15 smithi072 ceph-mon[29199]: purged_snaps scrub starts 2024-04-25T13:10:15.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:15 smithi072 ceph-mon[29199]: purged_snaps scrub ok 2024-04-25T13:10:15.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:15 smithi072 ceph-mon[29199]: osd.11 [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] boot 2024-04-25T13:10:15.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:15 smithi072 ceph-mon[29199]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:10:15.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:15 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:15.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:15 smithi072 ceph-mon[29199]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:15 smithi164 ceph-mon[32418]: purged_snaps scrub starts 2024-04-25T13:10:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:15 smithi164 ceph-mon[32418]: purged_snaps scrub ok 2024-04-25T13:10:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:15 smithi164 ceph-mon[32418]: osd.11 [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] boot 2024-04-25T13:10:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:15 smithi164 ceph-mon[32418]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:10:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:15 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:15 smithi164 ceph-mon[32418]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:15.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:15 smithi193 ceph-mon[32743]: purged_snaps scrub starts 2024-04-25T13:10:15.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:15 smithi193 ceph-mon[32743]: purged_snaps scrub ok 2024-04-25T13:10:15.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:15 smithi193 ceph-mon[32743]: osd.11 [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] boot 2024-04-25T13:10:15.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:15 smithi193 ceph-mon[32743]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:10:15.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:15 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:10:15.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:15 smithi193 ceph-mon[32743]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:10:15.983 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:10:16.221 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:10:16.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:10:16.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: Detected new or changed devices on smithi193 2024-04-25T13:10:16.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi193 to 4272M 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:16.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:16 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:10:16.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: Detected new or changed devices on smithi193 2024-04-25T13:10:16.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi193 to 4272M 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:16 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:10:16.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: Detected new or changed devices on smithi193 2024-04-25T13:10:16.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi193 to 4272M 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:16.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:16 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:17.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:17 smithi072 ceph-mon[29199]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:17.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:17 smithi164 ceph-mon[32418]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:17.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:17 smithi193 ceph-mon[32743]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.929+0000 7fd05a06e700 1 -- 172.21.15.72:0/579312167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054103d70 msgr2=0x7fd0541041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.929+0000 7fd05a06e700 1 --2- 172.21.15.72:0/579312167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054103d70 0x7fd0541041d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009990 tx=0x7fd03c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.929+0000 7fd05a06e700 1 -- 172.21.15.72:0/579312167 shutdown_connections 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.929+0000 7fd05a06e700 1 --2- 172.21.15.72:0/579312167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd054104710 0x7fd05410afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.929+0000 7fd05a06e700 1 --2- 172.21.15.72:0/579312167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd054100d90 0x7fd0541011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.929+0000 7fd05a06e700 1 --2- 172.21.15.72:0/579312167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054103d70 0x7fd0541041d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.929+0000 7fd05a06e700 1 -- 172.21.15.72:0/579312167 >> 172.21.15.72:0/579312167 conn(0x7fd0540fc6f0 msgr2=0x7fd0540feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.929+0000 7fd05a06e700 1 -- 172.21.15.72:0/579312167 shutdown_connections 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.929+0000 7fd05a06e700 1 -- 172.21.15.72:0/579312167 wait complete. 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd05a06e700 1 Processor -- start 2024-04-25T13:10:17.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd05a06e700 1 -- start start 2024-04-25T13:10:17.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd05a06e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd054100d90 0x7fd054198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:17.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd05a06e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd054103d70 0x7fd054199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:17.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd05a06e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054104710 0x7fd0541a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:17.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd05a06e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410e5d0 con 0x7fd054104710 2024-04-25T13:10:17.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd05a06e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410e450 con 0x7fd054100d90 2024-04-25T13:10:17.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd05a06e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410e750 con 0x7fd054103d70 2024-04-25T13:10:17.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd053fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054104710 0x7fd0541a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd053fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054104710 0x7fd0541a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55190/0 (socket says 172.21.15.72:55190) 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd053fff700 1 -- 172.21.15.72:0/2332870010 learned_addr learned my addr 172.21.15.72:0/2332870010 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.930+0000 7fd052ffd700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd054103d70 0x7fd054199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd0537fe700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd054100d90 0x7fd054198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd053fff700 1 -- 172.21.15.72:0/2332870010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd054103d70 msgr2=0x7fd054199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd053fff700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd054103d70 0x7fd054199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd053fff700 1 -- 172.21.15.72:0/2332870010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd054100d90 msgr2=0x7fd054198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd053fff700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd054100d90 0x7fd054198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd053fff700 1 -- 172.21.15.72:0/2332870010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0541a0880 con 0x7fd054104710 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd0537fe700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd054100d90 0x7fd054198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:17.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd053fff700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054104710 0x7fd0541a01b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b550 tx=0x7fd04c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:17.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd050ff9700 1 -- 172.21.15.72:0/2332870010 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c015020 con 0x7fd054104710 2024-04-25T13:10:17.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0541a0b70 con 0x7fd054104710 2024-04-25T13:10:17.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd050ff9700 1 -- 172.21.15.72:0/2332870010 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd04c011ce0 con 0x7fd054104710 2024-04-25T13:10:17.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd050ff9700 1 -- 172.21.15.72:0/2332870010 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c010dd0 con 0x7fd054104710 2024-04-25T13:10:17.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.931+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0541a10b0 con 0x7fd054104710 2024-04-25T13:10:17.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.932+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd054069730 con 0x7fd054104710 2024-04-25T13:10:17.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.934+0000 7fd050ff9700 1 -- 172.21.15.72:0/2332870010 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd04c018020 con 0x7fd054104710 2024-04-25T13:10:17.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.934+0000 7fd050ff9700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd0400a3c40 0x7fd0400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:17.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.935+0000 7fd0537fe700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd0400a3c40 0x7fd0400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:17.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.935+0000 7fd050ff9700 1 -- 172.21.15.72:0/2332870010 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd04c0cfbb0 con 0x7fd054104710 2024-04-25T13:10:17.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.935+0000 7fd0537fe700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd0400a3c40 0x7fd0400a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd05419a110 tx=0x7fd044009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:17.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:17.938+0000 7fd050ff9700 1 -- 172.21.15.72:0/2332870010 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd04c01f020 con 0x7fd054104710 2024-04-25T13:10:18.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.139+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd05402e190 con 0x7fd054104710 2024-04-25T13:10:18.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.139+0000 7fd050ff9700 1 -- 172.21.15.72:0/2332870010 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v84) ==== 74+0+133 (secure 0 0 0) 0x7fd04c09ca30 con 0x7fd054104710 2024-04-25T13:10:18.140 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:18.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.141+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd0400a3c40 msgr2=0x7fd0400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.141+0000 7fd05a06e700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd0400a3c40 0x7fd0400a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd05419a110 tx=0x7fd044009040 comp rx=0 tx=0).stop 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.141+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054104710 msgr2=0x7fd0541a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.141+0000 7fd05a06e700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054104710 0x7fd0541a01b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b550 tx=0x7fd04c00b860 comp rx=0 tx=0).stop 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.142+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 shutdown_connections 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.142+0000 7fd05a06e700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd0400a3c40 0x7fd0400a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.142+0000 7fd05a06e700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd054104710 0x7fd0541a01b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.142+0000 7fd05a06e700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd054100d90 0x7fd054198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.142+0000 7fd05a06e700 1 --2- 172.21.15.72:0/2332870010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd054103d70 0x7fd054199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.142+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 >> 172.21.15.72:0/2332870010 conn(0x7fd0540fc6f0 msgr2=0x7fd0541070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.142+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 shutdown_connections 2024-04-25T13:10:18.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:18.142+0000 7fd05a06e700 1 -- 172.21.15.72:0/2332870010 wait complete. 2024-04-25T13:10:18.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:18 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2332870010' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:10:18.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:18 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2332870010' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:10:18.851 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1714050614,"num_in_osds":12,"osd_in_since":1714050594,"num_remapped_pgs":0} 2024-04-25T13:10:18.851 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:10:18.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:18 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2332870010' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:10:19.085 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:10:19.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:19 smithi072 ceph-mon[29199]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:19.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:19 smithi164 ceph-mon[32418]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:19.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:19 smithi193 ceph-mon[32743]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.579+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1039415879 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4108a70 msgr2=0x7f22d4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.579+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1039415879 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4108a70 0x7f22d4108e50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009990 tx=0x7f22c4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.579+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1039415879 shutdown_connections 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.579+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1039415879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d4103480 0x7f22d410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.579+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1039415879 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4108a70 0x7f22d4108e50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.579+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1039415879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d4102ac0 0x7f22d4102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.579+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1039415879 >> 172.21.15.72:0/1039415879 conn(0x7f22d4079bb0 msgr2=0x7f22d4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.579+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1039415879 shutdown_connections 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.579+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1039415879 wait complete. 2024-04-25T13:10:20.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22dbb6e700 1 Processor -- start 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22dbb6e700 1 -- start start 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22dbb6e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4102ac0 0x7f22d41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22dbb6e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d4103480 0x7f22d4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22dbb6e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d4108a70 0x7f22d419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22dbb6e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f22d4112c80 con 0x7f22d4102ac0 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22dbb6e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22d4112b00 con 0x7f22d4108a70 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22dbb6e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22d4112e00 con 0x7f22d4103480 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22d990a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4102ac0 0x7f22d41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22d990a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4102ac0 0x7f22d41991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55200/0 (socket says 172.21.15.72:55200) 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22d990a700 1 -- 172.21.15.72:0/1840804562 learned_addr learned my addr 172.21.15.72:0/1840804562 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22d9109700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d4103480 0x7f22d4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.580+0000 7f22da10b700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d4108a70 0x7f22d419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:20.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22d990a700 1 -- 172.21.15.72:0/1840804562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d4103480 msgr2=0x7f22d4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22d990a700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d4103480 0x7f22d4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22d990a700 1 -- 172.21.15.72:0/1840804562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d4108a70 msgr2=0x7f22d419daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22d990a700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d4108a70 0x7f22d419daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22d990a700 1 -- 172.21.15.72:0/1840804562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d419e1e0 con 0x7f22d4102ac0 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22d990a700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4102ac0 0x7f22d41991d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22c4000c00 tx=0x7f22c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22caffd700 1 -- 172.21.15.72:0/1840804562 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4027070 con 0x7f22d4102ac0 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d419e470 con 0x7f22d4102ac0 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22caffd700 1 -- 172.21.15.72:0/1840804562 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c4003f20 con 0x7f22d4102ac0 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22caffd700 1 -- 172.21.15.72:0/1840804562 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4022ca0 con 0x7f22d4102ac0 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.581+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d41a5d50 con 0x7f22d4102ac0 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.582+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d4069730 con 0x7f22d4102ac0 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.583+0000 7f22caffd700 1 -- 172.21.15.72:0/1840804562 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f22c402a070 con 0x7f22d4102ac0 2024-04-25T13:10:20.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.583+0000 7f22caffd700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f22c00a3c40 0x7f22c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:20.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.584+0000 7f22d9109700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f22c00a3c40 0x7f22c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:20.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.584+0000 7f22caffd700 1 -- 172.21.15.72:0/1840804562 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f22c40d0dc0 con 0x7f22d4102ac0 2024-04-25T13:10:20.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.584+0000 7f22d9109700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f22c00a3c40 0x7f22c00a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f22d407a2d0 tx=0x7f22d0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:20.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.588+0000 7f22caffd700 1 -- 172.21.15.72:0/1840804562 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22c405eff0 con 0x7f22d4102ac0 2024-04-25T13:10:20.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.790+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22d402df80 con 0x7f22d4102ac0 2024-04-25T13:10:20.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.791+0000 7f22caffd700 1 -- 172.21.15.72:0/1840804562 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18480 (secure 0 0 0) 0x7f22c4013070 con 0x7f22d4102ac0 2024-04-25T13:10:20.791 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:20.792 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":84,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:10:15.483925+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:10:20.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.793+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f22c00a3c40 msgr2=0x7f22c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:20.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.793+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f22c00a3c40 0x7f22c00a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f22d407a2d0 tx=0x7f22d0009190 comp rx=0 tx=0).stop 2024-04-25T13:10:20.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.793+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4102ac0 msgr2=0x7f22d41991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:20.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.793+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4102ac0 0x7f22d41991d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22c4000c00 tx=0x7f22c40047b0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.794+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 shutdown_connections 2024-04-25T13:10:20.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.794+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d4108a70 0x7f22d419daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.794+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f22c00a3c40 0x7f22c00a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.794+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22d4102ac0 0x7f22d41991d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.794+0000 7f22dbb6e700 1 --2- 172.21.15.72:0/1840804562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d4103480 0x7f22d4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.794+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 >> 172.21.15.72:0/1840804562 conn(0x7f22d4079bb0 msgr2=0x7f22d41018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:20.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.794+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 shutdown_connections 2024-04-25T13:10:20.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:20.794+0000 7f22dbb6e700 1 -- 172.21.15.72:0/1840804562 wait complete. 2024-04-25T13:10:21.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:20 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1840804562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:10:21.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:20 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1840804562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:10:21.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:20 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1840804562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:10:21.492 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:06:33.150709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:10:21.492 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T13:10:21.732 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:10:22.199 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:21 smithi072 ceph-mon[29199]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:22.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:21 smithi193 ceph-mon[32743]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:22.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:21 smithi164 ceph-mon[32418]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.375+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/401148399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc06a750 msgr2=0x7fd3dc06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.375+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/401148399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc06a750 0x7fd3dc06abb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009990 tx=0x7fd3d0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.375+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/401148399 shutdown_connections 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.375+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/401148399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3dc06b0f0 0x7fd3dc10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.375+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/401148399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc108a30 0x7fd3dc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.375+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/401148399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc06a750 0x7fd3dc06abb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.375+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/401148399 >> 172.21.15.72:0/401148399 conn(0x7fd3dc0788f0 msgr2=0x7fd3dc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.375+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/401148399 shutdown_connections 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.375+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/401148399 wait complete. 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.376+0000 7fd3e0dbc700 1 Processor -- start 2024-04-25T13:10:23.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.376+0000 7fd3e0dbc700 1 -- start start 2024-04-25T13:10:23.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.376+0000 7fd3e0dbc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3dc06a750 0x7fd3dc1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:23.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.376+0000 7fd3e0dbc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc06b0f0 0x7fd3dc199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:23.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.376+0000 7fd3e0dbc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc108a30 0x7fd3dc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:23.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.376+0000 7fd3e0dbc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc112c20 con 0x7fd3dc108a30 2024-04-25T13:10:23.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.376+0000 7fd3e0dbc700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc112aa0 con 0x7fd3dc06b0f0 2024-04-25T13:10:23.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.376+0000 7fd3e0dbc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc112da0 con 0x7fd3dc06a750 2024-04-25T13:10:23.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.376+0000 7fd3dad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc108a30 0x7fd3dc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3dad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc108a30 0x7fd3dc19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55226/0 (socket says 172.21.15.72:55226) 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3dad9d700 1 -- 172.21.15.72:0/1590823279 learned_addr learned my addr 172.21.15.72:0/1590823279 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3d9d9b700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc06b0f0 0x7fd3dc199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3da59c700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3dc06a750 0x7fd3dc1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3dad9d700 1 -- 172.21.15.72:0/1590823279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3dc06a750 msgr2=0x7fd3dc1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3dad9d700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3dc06a750 0x7fd3dc1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3dad9d700 1 -- 172.21.15.72:0/1590823279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc06b0f0 msgr2=0x7fd3dc199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3dad9d700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc06b0f0 0x7fd3dc199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3dad9d700 1 -- 172.21.15.72:0/1590823279 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc19e0e0 con 0x7fd3dc108a30 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3da59c700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3dc06a750 0x7fd3dc1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3dad9d700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc108a30 0x7fd3dc19d9a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400ada0 tx=0x7fd3d400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3cb7fe700 1 -- 172.21.15.72:0/1590823279 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4017020 con 0x7fd3dc108a30 2024-04-25T13:10:23.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc19e3d0 con 0x7fd3dc108a30 2024-04-25T13:10:23.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3cb7fe700 1 -- 172.21.15.72:0/1590823279 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d400ce30 con 0x7fd3dc108a30 2024-04-25T13:10:23.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3cb7fe700 1 -- 172.21.15.72:0/1590823279 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4005350 con 0x7fd3dc108a30 2024-04-25T13:10:23.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.377+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc1a5c50 con 0x7fd3dc108a30 2024-04-25T13:10:23.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.378+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3dc069730 con 0x7fd3dc108a30 2024-04-25T13:10:23.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.379+0000 7fd3cb7fe700 1 -- 172.21.15.72:0/1590823279 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd3d400c980 con 0x7fd3dc108a30 2024-04-25T13:10:23.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.380+0000 7fd3cb7fe700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3c40a3c40 0x7fd3c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:23.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.380+0000 7fd3da59c700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3c40a3c40 0x7fd3c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:23.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.380+0000 7fd3cb7fe700 1 -- 172.21.15.72:0/1590823279 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd3d40cf6b0 con 0x7fd3dc108a30 2024-04-25T13:10:23.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.380+0000 7fd3da59c700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3c40a3c40 0x7fd3c40a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc005fd0 tx=0x7fd3cc00a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:23.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.385+0000 7fd3cb7fe700 1 -- 172.21.15.72:0/1590823279 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3d4021020 con 0x7fd3dc108a30 2024-04-25T13:10:23.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.581+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3dc19a430 con 0x7fd3dc108a30 2024-04-25T13:10:23.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.581+0000 7fd3cb7fe700 1 -- 172.21.15.72:0/1590823279 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fd3d409c5c0 con 0x7fd3dc108a30 2024-04-25T13:10:23.581 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-25T13:10:23.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3c40a3c40 msgr2=0x7fd3c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3c40a3c40 0x7fd3c40a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc005fd0 tx=0x7fd3cc00a5e0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc108a30 msgr2=0x7fd3dc19d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc108a30 0x7fd3dc19d9a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400ada0 tx=0x7fd3d400c330 comp rx=0 tx=0).stop 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 shutdown_connections 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3c40a3c40 0x7fd3c40a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3dc108a30 0x7fd3dc19d9a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3dc06a750 0x7fd3dc1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 --2- 172.21.15.72:0/1590823279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc06b0f0 0x7fd3dc199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 >> 172.21.15.72:0/1590823279 conn(0x7fd3dc0788f0 msgr2=0x7fd3dc0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 shutdown_connections 2024-04-25T13:10:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:23.583+0000 7fd3e0dbc700 1 -- 172.21.15.72:0/1590823279 wait complete. 2024-04-25T13:10:24.159 INFO:tasks.cephadm:Adding mds.a on smithi072 2024-04-25T13:10:24.159 INFO:tasks.cephadm:Adding mds.d on smithi072 2024-04-25T13:10:24.159 INFO:tasks.cephadm:Adding mds.g on smithi072 2024-04-25T13:10:24.159 INFO:tasks.cephadm:Adding mds.j on smithi072 2024-04-25T13:10:24.159 INFO:tasks.cephadm:Adding mds.b on smithi164 2024-04-25T13:10:24.159 INFO:tasks.cephadm:Adding mds.e on smithi164 2024-04-25T13:10:24.160 INFO:tasks.cephadm:Adding mds.h on smithi164 2024-04-25T13:10:24.160 INFO:tasks.cephadm:Adding mds.k on smithi164 2024-04-25T13:10:24.160 INFO:tasks.cephadm:Adding mds.c on smithi193 2024-04-25T13:10:24.160 INFO:tasks.cephadm:Adding mds.f on smithi193 2024-04-25T13:10:24.160 INFO:tasks.cephadm:Adding mds.i on smithi193 2024-04-25T13:10:24.160 INFO:tasks.cephadm:Adding mds.l on smithi193 2024-04-25T13:10:24.160 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2024-04-25T13:10:24.408 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:10:24.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:24 smithi193 ceph-mon[32743]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:24.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:24 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1590823279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:10:24.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:24 smithi072 ceph-mon[29199]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:24.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:24 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1590823279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:10:24.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:24 smithi164 ceph-mon[32418]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:24.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:24 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1590823279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:10:25.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 -- 172.21.15.193:0/3009581041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 msgr2=0x7fb394104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:25.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 --2- 172.21.15.193:0/3009581041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 0x7fb394104120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb384009990 tx=0x7fb384009ca0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 -- 172.21.15.193:0/3009581041 shutdown_connections 2024-04-25T13:10:25.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 --2- 172.21.15.193:0/3009581041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb394104660 0x7fb39410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 --2- 172.21.15.193:0/3009581041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb394102ac0 0x7fb394102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 --2- 172.21.15.193:0/3009581041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 0x7fb394104120 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 -- 172.21.15.193:0/3009581041 >> 172.21.15.193:0/3009581041 conn(0x7fb3940fe060 msgr2=0x7fb3941004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 -- 172.21.15.193:0/3009581041 shutdown_connections 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 -- 172.21.15.193:0/3009581041 wait complete. 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.977+0000 7fb39bd18700 1 Processor -- start 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb39bd18700 1 -- start start 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb39bd18700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb394102ac0 0x7fb394198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb39bd18700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 0x7fb394199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb39bd18700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb394104660 0x7fb3941a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb39bd18700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410e550 con 0x7fb394102ac0 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb39bd18700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410e3d0 con 0x7fb394104660 2024-04-25T13:10:25.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb39bd18700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410e6d0 con 0x7fb394103cc0 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb3992b3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 0x7fb394199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb3992b3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 0x7fb394199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:55812/0 (socket says 172.21.15.193:55812) 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb3992b3700 1 -- 172.21.15.193:0/1815074575 learned_addr learned my addr 172.21.15.193:0/1815074575 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb399ab4700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb394102ac0 0x7fb394198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb39a2b5700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb394104660 0x7fb3941a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb3992b3700 1 -- 172.21.15.193:0/1815074575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb394104660 msgr2=0x7fb3941a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb3992b3700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb394104660 0x7fb3941a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb3992b3700 1 -- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb394102ac0 msgr2=0x7fb394198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb3992b3700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb394102ac0 0x7fb394198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb3992b3700 1 -- 172.21.15.193:0/1815074575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3941a07a0 con 0x7fb394103cc0 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb39a2b5700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb394104660 0x7fb3941a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.978+0000 7fb3992b3700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 0x7fb394199080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb384000c00 tx=0x7fb38400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:25.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.979+0000 7fb38affd700 1 -- 172.21.15.193:0/1815074575 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384026070 con 0x7fb394103cc0 2024-04-25T13:10:25.982 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.979+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3941a0a30 con 0x7fb394103cc0 2024-04-25T13:10:25.982 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.979+0000 7fb38affd700 1 -- 172.21.15.193:0/1815074575 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb38400be90 con 0x7fb394103cc0 2024-04-25T13:10:25.982 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.979+0000 7fb38affd700 1 -- 172.21.15.193:0/1815074575 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384017650 con 0x7fb394103cc0 2024-04-25T13:10:25.982 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.979+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3941a0e80 con 0x7fb394103cc0 2024-04-25T13:10:25.982 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.980+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb394069730 con 0x7fb394103cc0 2024-04-25T13:10:25.983 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.981+0000 7fb38affd700 1 -- 172.21.15.193:0/1815074575 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb384029070 con 0x7fb394103cc0 2024-04-25T13:10:25.983 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.981+0000 7fb38affd700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb3800a3c40 0x7fb3800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:25.983 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.981+0000 7fb38affd700 1 -- 172.21.15.193:0/1815074575 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb3840d0fc0 con 0x7fb394103cc0 2024-04-25T13:10:25.983 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.981+0000 7fb399ab4700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb3800a3c40 0x7fb3800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:25.983 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.982+0000 7fb399ab4700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb3800a3c40 0x7fb3800a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb394103b20 tx=0x7fb38c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:25.985 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:25.986+0000 7fb38affd700 1 -- 172.21.15.193:0/1815074575 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb38405dff0 con 0x7fb394103cc0 2024-04-25T13:10:26.196 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.196+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) -- 0x7fb394062f80 con 0x7fb3800a3c40 2024-04-25T13:10:26.203 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.204+0000 7fb38affd700 1 -- 172.21.15.193:0/1815074575 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fb394062f80 con 0x7fb3800a3c40 2024-04-25T13:10:26.203 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb3800a3c40 msgr2=0x7fb3800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb3800a3c40 0x7fb3800a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb394103b20 tx=0x7fb38c00b040 comp rx=0 tx=0).stop 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 msgr2=0x7fb394199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 0x7fb394199080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb384000c00 tx=0x7fb38400f690 comp rx=0 tx=0).stop 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 shutdown_connections 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb394104660 0x7fb3941a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb3800a3c40 0x7fb3800a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb394102ac0 0x7fb394198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 --2- 172.21.15.193:0/1815074575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb394103cc0 0x7fb394199080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 >> 172.21.15.193:0/1815074575 conn(0x7fb3940fe060 msgr2=0x7fb394106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:26.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 shutdown_connections 2024-04-25T13:10:26.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T13:10:26.206+0000 7fb39bd18700 1 -- 172.21.15.193:0/1815074575 wait complete. 2024-04-25T13:10:26.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:26 smithi193 ceph-mon[32743]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:26.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:26 smithi072 ceph-mon[29199]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:26.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:26 smithi164 ceph-mon[32418]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:26.846 DEBUG:teuthology.orchestra.run.smithi072:mds.a> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T13:10:26.849 DEBUG:teuthology.orchestra.run.smithi072:mds.d> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T13:10:26.892 DEBUG:teuthology.orchestra.run.smithi072:mds.g> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T13:10:26.935 DEBUG:teuthology.orchestra.run.smithi072:mds.j> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T13:10:26.978 DEBUG:teuthology.orchestra.run.smithi164:mds.b> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T13:10:26.980 DEBUG:teuthology.orchestra.run.smithi164:mds.e> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T13:10:27.024 DEBUG:teuthology.orchestra.run.smithi164:mds.h> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T13:10:27.067 DEBUG:teuthology.orchestra.run.smithi164:mds.k> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T13:10:27.110 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T13:10:27.113 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T13:10:27.155 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T13:10:27.198 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T13:10:27.241 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T13:10:27.242 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:10:27.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:27.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: Saving service mds.all spec with placement smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-04-25T13:10:27.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:27.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:27.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:27.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:27.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:27.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:27.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:27.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:27 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.349+0000 7f627cf8a640 1 -- 172.21.15.72:0/2417622376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62781035d0 msgr2=0x7f62781039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.349+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2417622376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62781035d0 0x7f62781039f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6268009900 tx=0x7f626802a410 comp rx=0 tx=0).stop 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.349+0000 7f627cf8a640 1 -- 172.21.15.72:0/2417622376 shutdown_connections 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.349+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2417622376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6278100a00 0x7f627810a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.349+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2417622376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6278100040 0x7f62781004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.349+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2417622376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62781035d0 0x7f62781039f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.349+0000 7f627cf8a640 1 -- 172.21.15.72:0/2417622376 >> 172.21.15.72:0/2417622376 conn(0x7f62780fbbc0 msgr2=0x7f62780fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.349+0000 7f627cf8a640 1 -- 172.21.15.72:0/2417622376 shutdown_connections 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.349+0000 7f627cf8a640 1 -- 172.21.15.72:0/2417622376 wait complete. 2024-04-25T13:10:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627cf8a640 1 Processor -- start 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627cf8a640 1 -- start start 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627cf8a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6278100040 0x7f627819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627cf8a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6278100a00 0x7f627819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627cf8a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62781035d0 0x7f62781a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627cf8a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f627810db10 con 0x7f6278100040 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627cf8a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f627810d990 con 0x7f62781035d0 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627cf8a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f627810dc90 con 0x7f6278100a00 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6278100040 0x7f627819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6278100040 0x7f627819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46422/0 (socket says 172.21.15.72:46422) 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f627659c640 1 -- 172.21.15.72:0/2258601376 learned_addr learned my addr 172.21.15.72:0/2258601376 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.350+0000 7f6275d9b640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6278100a00 0x7f627819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f6276d9d640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62781035d0 0x7f62781a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f627659c640 1 -- 172.21.15.72:0/2258601376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6278100a00 msgr2=0x7f627819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f627659c640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6278100a00 0x7f627819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f627659c640 1 -- 172.21.15.72:0/2258601376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62781035d0 msgr2=0x7f62781a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f627659c640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62781035d0 0x7f62781a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f627659c640 1 -- 172.21.15.72:0/2258601376 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62781a2000 con 0x7f6278100040 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f6276d9d640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62781035d0 0x7f62781a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f627659c640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6278100040 0x7f627819a3e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6268006010 tx=0x7f62680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f62677fe640 1 -- 172.21.15.72:0/2258601376 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626803f030 con 0x7f6278100040 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f62677fe640 1 -- 172.21.15.72:0/2258601376 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6268003cb0 con 0x7f6278100040 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62781a2290 con 0x7f6278100040 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f62677fe640 1 -- 172.21.15.72:0/2258601376 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268035070 con 0x7f6278100040 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.351+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62781a2780 con 0x7f6278100040 2024-04-25T13:10:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.352+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62781a29e0 con 0x7f6278100040 2024-04-25T13:10:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.353+0000 7f62677fe640 1 -- 172.21.15.72:0/2258601376 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6268042030 con 0x7f6278100040 2024-04-25T13:10:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.354+0000 7f62677fe640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f62600a3bb0 0x7f62600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.354+0000 7f6275d9b640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f62600a3bb0 0x7f62600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.354+0000 7f62677fe640 1 -- 172.21.15.72:0/2258601376 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f62680f05a0 con 0x7f6278100040 2024-04-25T13:10:27.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.354+0000 7f6275d9b640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f62600a3bb0 0x7f62600a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f627819b8f0 tx=0x7f626c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:27.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.358+0000 7f62677fe640 1 -- 172.21.15.72:0/2258601376 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62680bd420 con 0x7f6278100040 2024-04-25T13:10:27.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:27.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: Saving service mds.all spec with placement smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-04-25T13:10:27.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:27.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:27.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:27.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:27.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:27.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:27.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:27.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:27 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:27.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:27.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: Saving service mds.all spec with placement smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-04-25T13:10:27.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:27.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:27.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:27.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:27.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:27.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:27.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:27.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:27 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:27.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.540+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62781a2bb0 con 0x7f6278100040 2024-04-25T13:10:27.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.541+0000 7f62677fe640 1 -- 172.21.15.72:0/2258601376 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18480 (secure 0 0 0) 0x7f62680bcd20 con 0x7f6278100040 2024-04-25T13:10:27.541 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:27.541 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":84,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:10:15.483925+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:10:27.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f62600a3bb0 msgr2=0x7f62600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f62600a3bb0 0x7f62600a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f627819b8f0 tx=0x7f626c00a2d0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6278100040 msgr2=0x7f627819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6278100040 0x7f627819a3e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6268006010 tx=0x7f62680042f0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 shutdown_connections 2024-04-25T13:10:27.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f62600a3bb0 0x7f62600a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62781035d0 0x7f62781a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6278100a00 0x7f627819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 --2- 172.21.15.72:0/2258601376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6278100040 0x7f627819a3e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 >> 172.21.15.72:0/2258601376 conn(0x7f62780fbbc0 msgr2=0x7f6278101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:27.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 shutdown_connections 2024-04-25T13:10:27.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.543+0000 7f627cf8a640 1 -- 172.21.15.72:0/2258601376 wait complete. 2024-04-25T13:10:27.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:06:33.150709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:10:27.554 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:10:27.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.655+0000 7fcb23e67640 1 -- 172.21.15.72:0/1801543383 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c1031a0 msgr2=0x7fcb1c103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.655+0000 7fcb23e67640 1 --2- 172.21.15.72:0/1801543383 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c1031a0 0x7fcb1c103620 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009900 tx=0x7fcb1802a410 comp rx=0 tx=0).stop 2024-04-25T13:10:27.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.655+0000 7fcb23e67640 1 -- 172.21.15.72:0/1801543383 shutdown_connections 2024-04-25T13:10:27.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.655+0000 7fcb23e67640 1 --2- 172.21.15.72:0/1801543383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb1c103b60 0x7fcb1c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.655+0000 7fcb23e67640 1 --2- 172.21.15.72:0/1801543383 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c1031a0 0x7fcb1c103620 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.655+0000 7fcb23e67640 1 --2- 172.21.15.72:0/1801543383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1c101fb0 0x7fcb1c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.655+0000 7fcb23e67640 1 -- 172.21.15.72:0/1801543383 >> 172.21.15.72:0/1801543383 conn(0x7fcb1c0fd520 msgr2=0x7fcb1c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.655+0000 7fcb23e67640 1 -- 172.21.15.72:0/1801543383 shutdown_connections 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.655+0000 7fcb23e67640 1 -- 172.21.15.72:0/1801543383 wait complete. 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb23e67640 1 Processor -- start 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb23e67640 1 -- start start 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb23e67640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c101fb0 0x7fcb1c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb23e67640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1c1031a0 0x7fcb1c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb23e67640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb1c103b60 0x7fcb1c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb23e67640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c10db80 con 0x7fcb1c101fb0 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb23e67640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c10da00 con 0x7fcb1c1031a0 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb23e67640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c10dd00 con 0x7fcb1c103b60 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb21c03640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c101fb0 0x7fcb1c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb21c03640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c101fb0 0x7fcb1c075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46434/0 (socket says 172.21.15.72:46434) 2024-04-25T13:10:27.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb21c03640 1 -- 172.21.15.72:0/2203243734 learned_addr learned my addr 172.21.15.72:0/2203243734 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:27.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb21402640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1c1031a0 0x7fcb1c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.656+0000 7fcb22404640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb1c103b60 0x7fcb1c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb21c03640 1 -- 172.21.15.72:0/2203243734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb1c103b60 msgr2=0x7fcb1c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb21c03640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb1c103b60 0x7fcb1c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb21c03640 1 -- 172.21.15.72:0/2203243734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1c1031a0 msgr2=0x7fcb1c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb21c03640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1c1031a0 0x7fcb1c074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb21c03640 1 -- 172.21.15.72:0/2203243734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c075110 con 0x7fcb1c101fb0 2024-04-25T13:10:27.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb21c03640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c101fb0 0x7fcb1c075d00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d950 tx=0x7fcb1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:27.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb0effd640 1 -- 172.21.15.72:0/2203243734 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10017020 con 0x7fcb1c101fb0 2024-04-25T13:10:27.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb0effd640 1 -- 172.21.15.72:0/2203243734 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb10009c30 con 0x7fcb1c101fb0 2024-04-25T13:10:27.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb0effd640 1 -- 172.21.15.72:0/2203243734 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000f6b0 con 0x7fcb1c101fb0 2024-04-25T13:10:27.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c1a2350 con 0x7fcb1c101fb0 2024-04-25T13:10:27.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.657+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c1a28c0 con 0x7fcb1c101fb0 2024-04-25T13:10:27.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.658+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1c109350 con 0x7fcb1c101fb0 2024-04-25T13:10:27.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.659+0000 7fcb0effd640 1 -- 172.21.15.72:0/2203243734 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcb1001a020 con 0x7fcb1c101fb0 2024-04-25T13:10:27.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.659+0000 7fcb0effd640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcb080a3c30 0x7fcb080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.660+0000 7fcb21402640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcb080a3c30 0x7fcb080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.660+0000 7fcb0effd640 1 -- 172.21.15.72:0/2203243734 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fcb100cfb30 con 0x7fcb1c101fb0 2024-04-25T13:10:27.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.660+0000 7fcb21402640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcb080a3c30 0x7fcb080a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcb18031500 tx=0x7fcb18005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:27.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.664+0000 7fcb0effd640 1 -- 172.21.15.72:0/2203243734 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb10009de0 con 0x7fcb1c101fb0 2024-04-25T13:10:27.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.841+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcb1c062b50 con 0x7fcb1c101fb0 2024-04-25T13:10:27.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.841+0000 7fcb0effd640 1 -- 172.21.15.72:0/2203243734 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fcb10096e20 con 0x7fcb1c101fb0 2024-04-25T13:10:27.841 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-25T13:10:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcb080a3c30 msgr2=0x7fcb080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcb080a3c30 0x7fcb080a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcb18031500 tx=0x7fcb18005a20 comp rx=0 tx=0).stop 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c101fb0 msgr2=0x7fcb1c075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c101fb0 0x7fcb1c075d00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d950 tx=0x7fcb1000de10 comp rx=0 tx=0).stop 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 shutdown_connections 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcb080a3c30 0x7fcb080a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb1c103b60 0x7fcb1c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1c1031a0 0x7fcb1c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 --2- 172.21.15.72:0/2203243734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb1c101fb0 0x7fcb1c075d00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 >> 172.21.15.72:0/2203243734 conn(0x7fcb1c0fd520 msgr2=0x7fcb1c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 shutdown_connections 2024-04-25T13:10:27.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.843+0000 7fcb23e67640 1 -- 172.21.15.72:0/2203243734 wait complete. 2024-04-25T13:10:27.854 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.953+0000 7f93fa443640 1 -- 172.21.15.72:0/3369864479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f41029a0 msgr2=0x7f93f41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.953+0000 7f93fa443640 1 --2- 172.21.15.72:0/3369864479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f41029a0 0x7f93f41092c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009ca0 tx=0x7f93ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.953+0000 7f93fa443640 1 -- 172.21.15.72:0/3369864479 shutdown_connections 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.953+0000 7f93fa443640 1 --2- 172.21.15.72:0/3369864479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f41029a0 0x7f93f41092c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.953+0000 7f93fa443640 1 --2- 172.21.15.72:0/3369864479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f4101fe0 0x7f93f4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.953+0000 7f93fa443640 1 --2- 172.21.15.72:0/3369864479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93f4100e90 0x7f93f41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.953+0000 7f93fa443640 1 -- 172.21.15.72:0/3369864479 >> 172.21.15.72:0/3369864479 conn(0x7f93f40fc400 msgr2=0x7f93f40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.953+0000 7f93fa443640 1 -- 172.21.15.72:0/3369864479 shutdown_connections 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.953+0000 7f93fa443640 1 -- 172.21.15.72:0/3369864479 wait complete. 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93fa443640 1 Processor -- start 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93fa443640 1 -- start start 2024-04-25T13:10:27.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93fa443640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f4100e90 0x7f93f4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93fa443640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93f4101fe0 0x7f93f4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93fa443640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41029a0 0x7f93f419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93fa443640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f93f410dbb0 con 0x7f93f4100e90 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93fa443640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93f410da30 con 0x7f93f41029a0 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93fa443640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f93f410dd30 con 0x7f93f4101fe0 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93f3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f4100e90 0x7f93f4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93f3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f4100e90 0x7f93f4198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46448/0 (socket says 172.21.15.72:46448) 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93f3fff640 1 -- 172.21.15.72:0/2880052422 learned_addr learned my addr 172.21.15.72:0/2880052422 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93f37fe640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93f4101fe0 0x7f93f4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93f3fff640 1 -- 172.21.15.72:0/2880052422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93f4101fe0 msgr2=0x7f93f4198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93f3fff640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93f4101fe0 0x7f93f4198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93f3fff640 1 -- 172.21.15.72:0/2880052422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41029a0 msgr2=0x7f93f419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:27.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.954+0000 7f93f89e0640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41029a0 0x7f93f419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.955+0000 7f93f3fff640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41029a0 0x7f93f419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.955+0000 7f93f3fff640 1 -- 172.21.15.72:0/2880052422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f419fe40 con 0x7f93f4100e90 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.955+0000 7f93f37fe640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93f4101fe0 0x7f93f4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.955+0000 7f93f3fff640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f4100e90 0x7f93f4198220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9f0 tx=0x7f93e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.955+0000 7f93f17fa640 1 -- 172.21.15.72:0/2880052422 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400cd80 con 0x7f93f4100e90 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.955+0000 7f93f17fa640 1 -- 172.21.15.72:0/2880052422 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e40054d0 con 0x7f93f4100e90 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.955+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f41a0130 con 0x7f93f4100e90 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.955+0000 7f93f17fa640 1 -- 172.21.15.72:0/2880052422 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4010690 con 0x7f93f4100e90 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.955+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f41a0700 con 0x7f93f4100e90 2024-04-25T13:10:27.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.956+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f41081a0 con 0x7f93f4100e90 2024-04-25T13:10:27.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.957+0000 7f93f17fa640 1 -- 172.21.15.72:0/2880052422 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f93e4004e90 con 0x7f93f4100e90 2024-04-25T13:10:27.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.958+0000 7f93f17fa640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f93dc0a3c30 0x7f93dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:27.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.958+0000 7f93f37fe640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f93dc0a3c30 0x7f93dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:27.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.958+0000 7f93f17fa640 1 -- 172.21.15.72:0/2880052422 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f93e40d0410 con 0x7f93f4100e90 2024-04-25T13:10:27.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.958+0000 7f93f37fe640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f93dc0a3c30 0x7f93dc0a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93f4199730 tx=0x7f93e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:27.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:27.962+0000 7f93f17fa640 1 -- 172.21.15.72:0/2880052422 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93e409d290 con 0x7f93f4100e90 2024-04-25T13:10:28.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.180+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93f402d980 con 0x7f93f4100e90 2024-04-25T13:10:28.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.181+0000 7f93f17fa640 1 -- 172.21.15.72:0/2880052422 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f93e409cb90 con 0x7f93f4100e90 2024-04-25T13:10:28.181 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:28.181 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T13:10:28.182 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 1 2024-04-25T13:10:28.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f93dc0a3c30 msgr2=0x7f93dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f93dc0a3c30 0x7f93dc0a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93f4199730 tx=0x7f93e8008040 comp rx=0 tx=0).stop 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f4100e90 msgr2=0x7f93f4198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f4100e90 0x7f93f4198220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9f0 tx=0x7f93e400edf0 comp rx=0 tx=0).stop 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 shutdown_connections 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f93dc0a3c30 0x7f93dc0a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41029a0 0x7f93f419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93f4101fe0 0x7f93f4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 --2- 172.21.15.72:0/2880052422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f4100e90 0x7f93f4198220 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 >> 172.21.15.72:0/2880052422 conn(0x7f93f40fc400 msgr2=0x7f93f4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 shutdown_connections 2024-04-25T13:10:28.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:28.183+0000 7f93fa443640 1 -- 172.21.15.72:0/2880052422 wait complete. 2024-04-25T13:10:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:28 smithi193 ceph-mon[32743]: Deploying daemon mds.l on smithi193 2024-04-25T13:10:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:28 smithi193 ceph-mon[32743]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:28 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2258601376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:10:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:28 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2203243734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:10:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:28 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2880052422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:28 smithi072 ceph-mon[29199]: Deploying daemon mds.l on smithi193 2024-04-25T13:10:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:28 smithi072 ceph-mon[29199]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:28 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2258601376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:10:28.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:28 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2203243734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:10:28.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:28 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2880052422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:28.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:28 smithi164 ceph-mon[32418]: Deploying daemon mds.l on smithi193 2024-04-25T13:10:28.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:28 smithi164 ceph-mon[32418]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:28.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:28 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2258601376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:10:28.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:28 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2203243734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:10:28.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:28 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2880052422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:28.981 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:10:28 smithi193 systemd[1]: Starting Ceph mds.l for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:29.255 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:10:28 smithi193 podman[62181]: 2024-04-25 13:10:28.89473181 +0000 UTC m=+0.017716389 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:29.255 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:10:29 smithi193 podman[62181]: 2024-04-25 13:10:29.01153421 +0000 UTC m=+0.134518818 container create 26563088b66f20a5ac9a83b2fe72db5b4aa3f3de95349d28b883e9fe1d3f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:10:29.256 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:10:29 smithi193 podman[62181]: 2024-04-25 13:10:29.230465518 +0000 UTC m=+0.353450084 container init 26563088b66f20a5ac9a83b2fe72db5b4aa3f3de95349d28b883e9fe1d3f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:10:29.256 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:10:29 smithi193 podman[62181]: 2024-04-25 13:10:29.233274268 +0000 UTC m=+0.356258843 container start 26563088b66f20a5ac9a83b2fe72db5b4aa3f3de95349d28b883e9fe1d3f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T13:10:29.730 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:10:29 smithi193 ceph-mds[62200]: -- 172.21.15.193:0/2373542367 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a30a425a40 con 0x55a30a3f9000 2024-04-25T13:10:29.731 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:10:29 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l[62196]: starting mds.l at 2024-04-25T13:10:29.731 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:10:29 smithi193 bash[62181]: 26563088b66f20a5ac9a83b2fe72db5b4aa3f3de95349d28b883e9fe1d3f2de3 2024-04-25T13:10:29.731 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:10:29 smithi193 systemd[1]: Started Ceph mds.l for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:30.195 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:30.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.305+0000 7fb672df7640 1 -- 172.21.15.72:0/3124675640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 msgr2=0x7fb66c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:30.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.305+0000 7fb672df7640 1 --2- 172.21.15.72:0/3124675640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 0x7fb66c1092c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb664009ca0 tx=0x7fb66402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.305+0000 7fb672df7640 1 -- 172.21.15.72:0/3124675640 shutdown_connections 2024-04-25T13:10:30.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 --2- 172.21.15.72:0/3124675640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 0x7fb66c1092c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 --2- 172.21.15.72:0/3124675640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb66c101fe0 0x7fb66c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 --2- 172.21.15.72:0/3124675640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c100e90 0x7fb66c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 -- 172.21.15.72:0/3124675640 >> 172.21.15.72:0/3124675640 conn(0x7fb66c0fc400 msgr2=0x7fb66c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 -- 172.21.15.72:0/3124675640 shutdown_connections 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 -- 172.21.15.72:0/3124675640 wait complete. 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 Processor -- start 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 -- start start 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb66c100e90 0x7fb66c197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c101fe0 0x7fb66c197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 0x7fb66c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c10c9d0 con 0x7fb66c1029a0 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c10c850 con 0x7fb66c101fe0 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb672df7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c10cb50 con 0x7fb66c100e90 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb671394640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 0x7fb66c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb671394640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 0x7fb66c19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46462/0 (socket says 172.21.15.72:46462) 2024-04-25T13:10:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb671394640 1 -- 172.21.15.72:0/297849852 learned_addr learned my addr 172.21.15.72:0/297849852 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.306+0000 7fb66bfff640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c101fe0 0x7fb66c197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb670b93640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb66c100e90 0x7fb66c197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb671394640 1 -- 172.21.15.72:0/297849852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb66c100e90 msgr2=0x7fb66c197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb671394640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb66c100e90 0x7fb66c197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb671394640 1 -- 172.21.15.72:0/297849852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c101fe0 msgr2=0x7fb66c197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb671394640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c101fe0 0x7fb66c197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb671394640 1 -- 172.21.15.72:0/297849852 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66c19ec60 con 0x7fb66c1029a0 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb670b93640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb66c100e90 0x7fb66c197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb671394640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 0x7fb66c19e5f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb6640097b0 tx=0x7fb664003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb669ffb640 1 -- 172.21.15.72:0/297849852 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66404a070 con 0x7fb66c1029a0 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb669ffb640 1 -- 172.21.15.72:0/297849852 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6640317c0 con 0x7fb66c1029a0 2024-04-25T13:10:30.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66c0690e0 con 0x7fb66c1029a0 2024-04-25T13:10:30.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb669ffb640 1 -- 172.21.15.72:0/297849852 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66402fbd0 con 0x7fb66c1029a0 2024-04-25T13:10:30.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.307+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66c069630 con 0x7fb66c1029a0 2024-04-25T13:10:30.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.308+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66c068d70 con 0x7fb66c1029a0 2024-04-25T13:10:30.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.309+0000 7fb669ffb640 1 -- 172.21.15.72:0/297849852 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb66404d070 con 0x7fb66c1029a0 2024-04-25T13:10:30.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.310+0000 7fb669ffb640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb65c0a3bb0 0x7fb65c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:30.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.310+0000 7fb670b93640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb65c0a3bb0 0x7fb65c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:30.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.310+0000 7fb669ffb640 1 -- 172.21.15.72:0/297849852 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb6640f2a10 con 0x7fb66c1029a0 2024-04-25T13:10:30.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.310+0000 7fb670b93640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb65c0a3bb0 0x7fb65c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb660009b40 tx=0x7fb660005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:30.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.314+0000 7fb669ffb640 1 -- 172.21.15.72:0/297849852 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb664081ff0 con 0x7fb66c1029a0 2024-04-25T13:10:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.193:6832/3064383723,v1:172.21.15.193:6833/3064383723] up:boot 2024-04-25T13:10:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: fsmap 1 up:standby 2024-04-25T13:10:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:10:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:30 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:30.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.528+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb66c107df0 con 0x7fb66c1029a0 2024-04-25T13:10:30.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.528+0000 7fb669ffb640 1 -- 172.21.15.72:0/297849852 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fb6640bf890 con 0x7fb66c1029a0 2024-04-25T13:10:30.528 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:30.528 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:10:30.529 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 2 2024-04-25T13:10:30.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb65c0a3bb0 msgr2=0x7fb65c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:30.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb65c0a3bb0 0x7fb65c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb660009b40 tx=0x7fb660005e50 comp rx=0 tx=0).stop 2024-04-25T13:10:30.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 msgr2=0x7fb66c19e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 0x7fb66c19e5f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb6640097b0 tx=0x7fb664003ba0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 shutdown_connections 2024-04-25T13:10:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb65c0a3bb0 0x7fb65c0a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb66c1029a0 0x7fb66c19e5f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c101fe0 0x7fb66c197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 --2- 172.21.15.72:0/297849852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb66c100e90 0x7fb66c197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 >> 172.21.15.72:0/297849852 conn(0x7fb66c0fc400 msgr2=0x7fb66c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 shutdown_connections 2024-04-25T13:10:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:30.530+0000 7fb672df7640 1 -- 172.21.15.72:0/297849852 wait complete. 2024-04-25T13:10:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.193:6832/3064383723,v1:172.21.15.193:6833/3064383723] up:boot 2024-04-25T13:10:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: fsmap 1 up:standby 2024-04-25T13:10:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:10:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:30.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:30.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:30.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:30.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:30 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.193:6832/3064383723,v1:172.21.15.193:6833/3064383723] up:boot 2024-04-25T13:10:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: fsmap 1 up:standby 2024-04-25T13:10:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:10:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:30.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:30.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:30 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:31.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:31 smithi072 ceph-mon[29199]: Deploying daemon mds.k on smithi164 2024-04-25T13:10:31.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:31 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/297849852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:31.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:31 smithi164 ceph-mon[32418]: Deploying daemon mds.k on smithi164 2024-04-25T13:10:31.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:31 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/297849852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:31.557 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:10:31 smithi164 systemd[1]: Starting Ceph mds.k for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:31.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:31 smithi193 ceph-mon[32743]: Deploying daemon mds.k on smithi164 2024-04-25T13:10:31.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:31 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/297849852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:31.942 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:10:31 smithi164 podman[61287]: 2024-04-25 13:10:31.590102378 +0000 UTC m=+0.018315082 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:31.942 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:10:31 smithi164 podman[61287]: 2024-04-25 13:10:31.690484613 +0000 UTC m=+0.118697302 container create 4ac470d933d855c9a11070aae3a5310303e0fadcbbee84c4f9a46e79126a6d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T13:10:31.942 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:10:31 smithi164 podman[61287]: 2024-04-25 13:10:31.917699618 +0000 UTC m=+0.345912307 container init 4ac470d933d855c9a11070aae3a5310303e0fadcbbee84c4f9a46e79126a6d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T13:10:31.942 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:10:31 smithi164 podman[61287]: 2024-04-25 13:10:31.920141064 +0000 UTC m=+0.348353756 container start 4ac470d933d855c9a11070aae3a5310303e0fadcbbee84c4f9a46e79126a6d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:10:31.942 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:10:31 smithi164 ceph-mds[61306]: -- 172.21.15.164:0/3472415737 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4f318da40 con 0x55d4f3161000 2024-04-25T13:10:32.278 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:10:31 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k[61302]: starting mds.k at 2024-04-25T13:10:32.278 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:10:32 smithi164 bash[61287]: 4ac470d933d855c9a11070aae3a5310303e0fadcbbee84c4f9a46e79126a6d20 2024-04-25T13:10:32.278 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:10:32 smithi164 systemd[1]: Started Ceph mds.k for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:32.543 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:32 smithi164 ceph-mon[32418]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:32 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:32.564 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:32 smithi072 ceph-mon[29199]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:32.564 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:32.564 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:32.564 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:32.565 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:32.565 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:32.565 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:32 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.647+0000 7f0a7789e640 1 -- 172.21.15.72:0/3144617760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 msgr2=0x7f0a701012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.647+0000 7f0a7789e640 1 --2- 172.21.15.72:0/3144617760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 0x7f0a701012c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009900 tx=0x7f0a6c02a410 comp rx=0 tx=0).stop 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.647+0000 7f0a7789e640 1 -- 172.21.15.72:0/3144617760 shutdown_connections 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.647+0000 7f0a7789e640 1 --2- 172.21.15.72:0/3144617760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a701029b0 0x7f0a701092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.647+0000 7f0a7789e640 1 --2- 172.21.15.72:0/3144617760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a70101ff0 0x7f0a70102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.647+0000 7f0a7789e640 1 --2- 172.21.15.72:0/3144617760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 0x7f0a701012c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.647+0000 7f0a7789e640 1 -- 172.21.15.72:0/3144617760 >> 172.21.15.72:0/3144617760 conn(0x7f0a700fc430 msgr2=0x7f0a700fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.647+0000 7f0a7789e640 1 -- 172.21.15.72:0/3144617760 shutdown_connections 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.647+0000 7f0a7789e640 1 -- 172.21.15.72:0/3144617760 wait complete. 2024-04-25T13:10:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7789e640 1 Processor -- start 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7789e640 1 -- start start 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7789e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 0x7f0a70197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7789e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a70101ff0 0x7f0a701976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7789e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a701029b0 0x7f0a7019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7789e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7010ca50 con 0x7f0a70100ea0 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7789e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7010c8d0 con 0x7f0a701029b0 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7789e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7010cbd0 con 0x7f0a70101ff0 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7563a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 0x7f0a70197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7563a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 0x7f0a70197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46486/0 (socket says 172.21.15.72:46486) 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7563a640 1 -- 172.21.15.72:0/983993564 learned_addr learned my addr 172.21.15.72:0/983993564 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7563a640 1 -- 172.21.15.72:0/983993564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a70101ff0 msgr2=0x7f0a701976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7563a640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a70101ff0 0x7f0a701976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7563a640 1 -- 172.21.15.72:0/983993564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a701029b0 msgr2=0x7f0a7019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:32.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7563a640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a701029b0 0x7f0a7019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7563a640 1 -- 172.21.15.72:0/983993564 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7019ed90 con 0x7f0a70100ea0 2024-04-25T13:10:32.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a7563a640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 0x7f0a70197170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c031500 tx=0x7f0a6c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:32.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.648+0000 7f0a667fc640 1 -- 172.21.15.72:0/983993564 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c03f030 con 0x7f0a70100ea0 2024-04-25T13:10:32.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.649+0000 7f0a667fc640 1 -- 172.21.15.72:0/983993564 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a6c003800 con 0x7f0a70100ea0 2024-04-25T13:10:32.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.649+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7019f020 con 0x7f0a70100ea0 2024-04-25T13:10:32.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.649+0000 7f0a667fc640 1 -- 172.21.15.72:0/983993564 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c035070 con 0x7f0a70100ea0 2024-04-25T13:10:32.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.649+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7019f590 con 0x7f0a70100ea0 2024-04-25T13:10:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.649+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a70108280 con 0x7f0a70100ea0 2024-04-25T13:10:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.650+0000 7f0a667fc640 1 -- 172.21.15.72:0/983993564 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0a6c0039b0 con 0x7f0a70100ea0 2024-04-25T13:10:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.651+0000 7f0a667fc640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0a5c0a3bb0 0x7f0a5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.651+0000 7f0a74e39640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0a5c0a3bb0 0x7f0a5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.651+0000 7f0a667fc640 1 -- 172.21.15.72:0/983993564 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f0a6c0f1080 con 0x7f0a70100ea0 2024-04-25T13:10:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.651+0000 7f0a74e39640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0a5c0a3bb0 0x7f0a5c0a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a70198680 tx=0x7f0a60005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:32.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.655+0000 7f0a667fc640 1 -- 172.21.15.72:0/983993564 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a6c0bdf00 con 0x7f0a70100ea0 2024-04-25T13:10:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:32 smithi193 ceph-mon[32743]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:32 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:32.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.877+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a7002d5c0 con 0x7f0a70100ea0 2024-04-25T13:10:32.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.878+0000 7f0a667fc640 1 -- 172.21.15.72:0/983993564 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f0a6c0bd800 con 0x7f0a70100ea0 2024-04-25T13:10:32.878 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:32.878 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:10:32.878 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 3 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0a5c0a3bb0 msgr2=0x7f0a5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0a5c0a3bb0 0x7f0a5c0a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a70198680 tx=0x7f0a60005c10 comp rx=0 tx=0).stop 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 msgr2=0x7f0a70197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 0x7f0a70197170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c031500 tx=0x7f0a6c0043c0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 shutdown_connections 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0a5c0a3bb0 0x7f0a5c0a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a701029b0 0x7f0a7019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a70101ff0 0x7f0a701976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 --2- 172.21.15.72:0/983993564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a70100ea0 0x7f0a70197170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 >> 172.21.15.72:0/983993564 conn(0x7f0a700fc430 msgr2=0x7f0a70105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:32.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 shutdown_connections 2024-04-25T13:10:32.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:32.880+0000 7f0a7789e640 1 -- 172.21.15.72:0/983993564 wait complete. 2024-04-25T13:10:33.320 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:33 smithi164 ceph-mon[32418]: Deploying daemon mds.j on smithi072 2024-04-25T13:10:33.320 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:33 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.164:6832/1516437385,v1:172.21.15.164:6833/1516437385] up:boot 2024-04-25T13:10:33.320 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:33 smithi164 ceph-mon[32418]: fsmap 2 up:standby 2024-04-25T13:10:33.321 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:33 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:10:33.321 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:33 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/983993564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:33.716 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:33 smithi072 ceph-mon[29199]: Deploying daemon mds.j on smithi072 2024-04-25T13:10:33.717 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:33 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.164:6832/1516437385,v1:172.21.15.164:6833/1516437385] up:boot 2024-04-25T13:10:33.717 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:33 smithi072 ceph-mon[29199]: fsmap 2 up:standby 2024-04-25T13:10:33.717 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:33 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:10:33.717 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:33 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/983993564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:33.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:33 smithi193 ceph-mon[32743]: Deploying daemon mds.j on smithi072 2024-04-25T13:10:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:33 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.164:6832/1516437385,v1:172.21.15.164:6833/1516437385] up:boot 2024-04-25T13:10:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:33 smithi193 ceph-mon[32743]: fsmap 2 up:standby 2024-04-25T13:10:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:33 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:10:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:33 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/983993564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:34.274 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:10:34 smithi072 systemd[1]: Starting Ceph mds.j for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:34.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:34 smithi164 ceph-mon[32418]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:34.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:34 smithi072 ceph-mon[29199]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:34.595 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:10:34 smithi072 podman[69998]: 2024-04-25 13:10:34.270033372 +0000 UTC m=+0.018171495 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:34.595 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:10:34 smithi072 podman[69998]: 2024-04-25 13:10:34.383337381 +0000 UTC m=+0.131475506 container create a009131c2d33c2f50d2c10846c50fda0a30f7240d080a9d02842b059a5c412d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T13:10:34.595 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:10:34 smithi072 podman[69998]: 2024-04-25 13:10:34.596255144 +0000 UTC m=+0.344393259 container init a009131c2d33c2f50d2c10846c50fda0a30f7240d080a9d02842b059a5c412d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T13:10:34.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:34 smithi193 ceph-mon[32743]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:34.891 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:34.913 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:10:34 smithi072 podman[69998]: 2024-04-25 13:10:34.598976599 +0000 UTC m=+0.347114715 container start a009131c2d33c2f50d2c10846c50fda0a30f7240d080a9d02842b059a5c412d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:10:34.913 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:10:34 smithi072 ceph-mds[70017]: -- 172.21.15.72:0/3174276765 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557627db7a40 con 0x557627d8b000 2024-04-25T13:10:34.913 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:10:34 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j[70013]: starting mds.j at 2024-04-25T13:10:34.913 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:10:34 smithi072 bash[69998]: a009131c2d33c2f50d2c10846c50fda0a30f7240d080a9d02842b059a5c412d0 2024-04-25T13:10:34.914 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:10:34 smithi072 systemd[1]: Started Ceph mds.j for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:35.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.003+0000 7fe86a8e1640 1 -- 172.21.15.72:0/638880999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864105340 msgr2=0x7fe864107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.003+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/638880999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864105340 0x7fe864107730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe858009900 tx=0x7fe85802a410 comp rx=0 tx=0).stop 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.003+0000 7fe86a8e1640 1 -- 172.21.15.72:0/638880999 shutdown_connections 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.003+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/638880999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864107c70 0x7fe86410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.003+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/638880999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864105340 0x7fe864107730 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.003+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/638880999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe864102a10 0x7fe864104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.003+0000 7fe86a8e1640 1 -- 172.21.15.72:0/638880999 >> 172.21.15.72:0/638880999 conn(0x7fe8640fc400 msgr2=0x7fe8640fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.003+0000 7fe86a8e1640 1 -- 172.21.15.72:0/638880999 shutdown_connections 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.003+0000 7fe86a8e1640 1 -- 172.21.15.72:0/638880999 wait complete. 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe86a8e1640 1 Processor -- start 2024-04-25T13:10:35.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe86a8e1640 1 -- start start 2024-04-25T13:10:35.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe86a8e1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864102a10 0x7fe864102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:35.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe86a8e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105340 0x7fe8641009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe86a8e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe864107c70 0x7fe864100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe86a8e1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe86410dac0 con 0x7fe864102a10 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe86a8e1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe86410d940 con 0x7fe864105340 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe86a8e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe86410dc40 con 0x7fe864107c70 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe863fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864102a10 0x7fe864102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe863fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864102a10 0x7fe864102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46510/0 (socket says 172.21.15.72:46510) 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe863fff640 1 -- 172.21.15.72:0/1250842776 learned_addr learned my addr 172.21.15.72:0/1250842776 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe8637fe640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105340 0x7fe8641009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.004+0000 7fe868e7e640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe864107c70 0x7fe864100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe863fff640 1 -- 172.21.15.72:0/1250842776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe864107c70 msgr2=0x7fe864100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe863fff640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe864107c70 0x7fe864100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe863fff640 1 -- 172.21.15.72:0/1250842776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105340 msgr2=0x7fe8641009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:35.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe863fff640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105340 0x7fe8641009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe863fff640 1 -- 172.21.15.72:0/1250842776 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe864101790 con 0x7fe864102a10 2024-04-25T13:10:35.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe863fff640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864102a10 0x7fe864102380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe85400d770 tx=0x7fe85400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:35.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe8617fa640 1 -- 172.21.15.72:0/1250842776 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854017070 con 0x7fe864102a10 2024-04-25T13:10:35.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8641a2380 con 0x7fe864102a10 2024-04-25T13:10:35.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe8617fa640 1 -- 172.21.15.72:0/1250842776 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8540054d0 con 0x7fe864102a10 2024-04-25T13:10:35.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe8617fa640 1 -- 172.21.15.72:0/1250842776 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854004c30 con 0x7fe864102a10 2024-04-25T13:10:35.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.005+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8641a2850 con 0x7fe864102a10 2024-04-25T13:10:35.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.006+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8640fdef0 con 0x7fe864102a10 2024-04-25T13:10:35.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.007+0000 7fe8617fa640 1 -- 172.21.15.72:0/1250842776 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe854004e50 con 0x7fe864102a10 2024-04-25T13:10:35.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.007+0000 7fe8617fa640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe84c0a3bb0 0x7fe84c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:35.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.008+0000 7fe8637fe640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe84c0a3bb0 0x7fe84c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:35.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.008+0000 7fe8617fa640 1 -- 172.21.15.72:0/1250842776 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe8540cf990 con 0x7fe864102a10 2024-04-25T13:10:35.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.008+0000 7fe8637fe640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe84c0a3bb0 0x7fe84c0a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe858009230 tx=0x7fe8580314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:35.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.012+0000 7fe8617fa640 1 -- 172.21.15.72:0/1250842776 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe85409c1c0 con 0x7fe864102a10 2024-04-25T13:10:35.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.252+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe864062b50 con 0x7fe864102a10 2024-04-25T13:10:35.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.252+0000 7fe8617fa640 1 -- 172.21.15.72:0/1250842776 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fe8540a04c0 con 0x7fe864102a10 2024-04-25T13:10:35.252 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:35.252 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:10:35.253 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 3 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.254+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe84c0a3bb0 msgr2=0x7fe84c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.254+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe84c0a3bb0 0x7fe84c0a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe858009230 tx=0x7fe8580314a0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.254+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864102a10 msgr2=0x7fe864102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.254+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864102a10 0x7fe864102380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe85400d770 tx=0x7fe85400dc30 comp rx=0 tx=0).stop 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.254+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 shutdown_connections 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.255+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe84c0a3bb0 0x7fe84c0a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.255+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe864107c70 0x7fe864100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.255+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105340 0x7fe8641009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.255+0000 7fe86a8e1640 1 --2- 172.21.15.72:0/1250842776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe864102a10 0x7fe864102380 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.255+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 >> 172.21.15.72:0/1250842776 conn(0x7fe8640fc400 msgr2=0x7fe864103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:35.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.255+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 shutdown_connections 2024-04-25T13:10:35.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:35.255+0000 7fe86a8e1640 1 -- 172.21.15.72:0/1250842776 wait complete. 2024-04-25T13:10:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:36.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:36.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:36.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:36.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:36.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: Deploying daemon mds.d on smithi072 2024-04-25T13:10:36.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:36.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1250842776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:36.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.72:6834/3145624420,v1:172.21.15.72:6835/3145624420] up:boot 2024-04-25T13:10:36.026 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: fsmap 3 up:standby 2024-04-25T13:10:36.026 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:35 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:10:36.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:36.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: Deploying daemon mds.d on smithi072 2024-04-25T13:10:36.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:36.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1250842776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:36.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.72:6834/3145624420,v1:172.21.15.72:6835/3145624420] up:boot 2024-04-25T13:10:36.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: fsmap 3 up:standby 2024-04-25T13:10:36.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:35 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:10:36.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:36.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:36.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:36.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:36.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:36.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:36.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: Deploying daemon mds.d on smithi072 2024-04-25T13:10:36.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:36.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1250842776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:36.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.72:6834/3145624420,v1:172.21.15.72:6835/3145624420] up:boot 2024-04-25T13:10:36.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: fsmap 3 up:standby 2024-04-25T13:10:36.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:35 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:10:37.024 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:10:36 smithi072 systemd[1]: Starting Ceph mds.d for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:37.267 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:37.344 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:10:37 smithi072 podman[70348]: 2024-04-25 13:10:36.980193601 +0000 UTC m=+0.018055301 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:37.344 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:10:37 smithi072 podman[70348]: 2024-04-25 13:10:37.09656445 +0000 UTC m=+0.134426125 container create d4657a749527419aa839ba1c06d6b0c99f41c16ac0c67811f35ca0b90374ae8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T13:10:37.344 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:10:37 smithi072 podman[70348]: 2024-04-25 13:10:37.34512577 +0000 UTC m=+0.382987460 container init d4657a749527419aa839ba1c06d6b0c99f41c16ac0c67811f35ca0b90374ae8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:10:37.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 -- 172.21.15.72:0/730998294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f301029a0 msgr2=0x7f7f301092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:37.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 --2- 172.21.15.72:0/730998294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f301029a0 0x7f7f301092c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009ca0 tx=0x7f7f2802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 -- 172.21.15.72:0/730998294 shutdown_connections 2024-04-25T13:10:37.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 --2- 172.21.15.72:0/730998294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f301029a0 0x7f7f301092c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 --2- 172.21.15.72:0/730998294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f30101fe0 0x7f7f30102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 --2- 172.21.15.72:0/730998294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f30100e90 0x7f7f301012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 -- 172.21.15.72:0/730998294 >> 172.21.15.72:0/730998294 conn(0x7f7f300fc400 msgr2=0x7f7f300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 -- 172.21.15.72:0/730998294 shutdown_connections 2024-04-25T13:10:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 -- 172.21.15.72:0/730998294 wait complete. 2024-04-25T13:10:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.372+0000 7f7f3527c640 1 Processor -- start 2024-04-25T13:10:37.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f3527c640 1 -- start start 2024-04-25T13:10:37.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f3527c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f30100e90 0x7f7f30069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f3527c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f30101fe0 0x7f7f3019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f3527c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f301029a0 0x7f7f3019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f3527c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3010ca30 con 0x7f7f30101fe0 2024-04-25T13:10:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f3527c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3010c8b0 con 0x7f7f30100e90 2024-04-25T13:10:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f3527c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3010cbb0 con 0x7f7f301029a0 2024-04-25T13:10:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2e7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f30101fe0 0x7f7f3019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2e7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f30101fe0 0x7f7f3019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45402/0 (socket says 172.21.15.72:45402) 2024-04-25T13:10:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2e7fc640 1 -- 172.21.15.72:0/2134450761 learned_addr learned my addr 172.21.15.72:0/2134450761 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2effd640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f30100e90 0x7f7f30069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2f7fe640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f301029a0 0x7f7f3019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2e7fc640 1 -- 172.21.15.72:0/2134450761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f301029a0 msgr2=0x7f7f3019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2e7fc640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f301029a0 0x7f7f3019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2e7fc640 1 -- 172.21.15.72:0/2134450761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f30100e90 msgr2=0x7f7f30069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2e7fc640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f30100e90 0x7f7f30069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.373+0000 7f7f2e7fc640 1 -- 172.21.15.72:0/2134450761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3019fd70 con 0x7f7f30101fe0 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.374+0000 7f7f2e7fc640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f30101fe0 0x7f7f3019ce40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c970 tx=0x7f7f2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.374+0000 7f7f17fff640 1 -- 172.21.15.72:0/2134450761 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20004220 con 0x7f7f30101fe0 2024-04-25T13:10:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.374+0000 7f7f17fff640 1 -- 172.21.15.72:0/2134450761 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f200043c0 con 0x7f7f30101fe0 2024-04-25T13:10:37.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.374+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f301a0060 con 0x7f7f30101fe0 2024-04-25T13:10:37.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.374+0000 7f7f17fff640 1 -- 172.21.15.72:0/2134450761 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20004a00 con 0x7f7f30101fe0 2024-04-25T13:10:37.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.374+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f301a0560 con 0x7f7f30101fe0 2024-04-25T13:10:37.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.375+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f301081a0 con 0x7f7f30101fe0 2024-04-25T13:10:37.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.376+0000 7f7f17fff640 1 -- 172.21.15.72:0/2134450761 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7f200079b0 con 0x7f7f30101fe0 2024-04-25T13:10:37.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.377+0000 7f7f17fff640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7f180a3bb0 0x7f7f180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:37.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.377+0000 7f7f17fff640 1 -- 172.21.15.72:0/2134450761 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f7f200d0050 con 0x7f7f30101fe0 2024-04-25T13:10:37.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.377+0000 7f7f2effd640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7f180a3bb0 0x7f7f180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:37.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.377+0000 7f7f2effd640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7f180a3bb0 0x7f7f180a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400bbe0 tx=0x7f7f2400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:37.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.380+0000 7f7f17fff640 1 -- 172.21.15.72:0/2134450761 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f20014030 con 0x7f7f30101fe0 2024-04-25T13:10:37.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.597+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f3002d5c0 con 0x7f7f30101fe0 2024-04-25T13:10:37.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.598+0000 7f7f17fff640 1 -- 172.21.15.72:0/2134450761 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f7f2009c880 con 0x7f7f30101fe0 2024-04-25T13:10:37.598 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:37.598 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:10:37.598 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 4 2024-04-25T13:10:37.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.599+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7f180a3bb0 msgr2=0x7f7f180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:37.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.599+0000 7f7f3527c640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7f180a3bb0 0x7f7f180a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400bbe0 tx=0x7f7f2400b390 comp rx=0 tx=0).stop 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.599+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f30101fe0 msgr2=0x7f7f3019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.599+0000 7f7f3527c640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f30101fe0 0x7f7f3019ce40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c970 tx=0x7f7f2000ce30 comp rx=0 tx=0).stop 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.599+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 shutdown_connections 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.600+0000 7f7f3527c640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7f180a3bb0 0x7f7f180a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.600+0000 7f7f3527c640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f301029a0 0x7f7f3019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.600+0000 7f7f3527c640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f30101fe0 0x7f7f3019ce40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.600+0000 7f7f3527c640 1 --2- 172.21.15.72:0/2134450761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f30100e90 0x7f7f30069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.600+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 >> 172.21.15.72:0/2134450761 conn(0x7f7f300fc400 msgr2=0x7f7f30105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.600+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 shutdown_connections 2024-04-25T13:10:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:37.600+0000 7f7f3527c640 1 -- 172.21.15.72:0/2134450761 wait complete. 2024-04-25T13:10:37.610 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:10:37 smithi072 podman[70348]: 2024-04-25 13:10:37.349591371 +0000 UTC m=+0.387453044 container start d4657a749527419aa839ba1c06d6b0c99f41c16ac0c67811f35ca0b90374ae8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-25T13:10:37.610 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:10:37 smithi072 ceph-mds[70387]: -- 172.21.15.72:0/997296250 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db3fd5ba40 con 0x55db3fd2ec00 2024-04-25T13:10:37.610 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:10:37 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-d[70363]: starting mds.d at 2024-04-25T13:10:37.610 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:10:37 smithi072 bash[70348]: d4657a749527419aa839ba1c06d6b0c99f41c16ac0c67811f35ca0b90374ae8b 2024-04-25T13:10:37.610 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:10:37 smithi072 systemd[1]: Started Ceph mds.d for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:38 smithi193 ceph-mon[32743]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:38 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2134450761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:38.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:38 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:38.503 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:38 smithi164 ceph-mon[32418]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:38.503 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:38 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2134450761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:38.503 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:38.503 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:38.504 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:38.504 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:38.504 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:38.504 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:38 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:38.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:38 smithi072 ceph-mon[29199]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:38.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:38 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2134450761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:38.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:38.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:38.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:38.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:38.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:38.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:38 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:39.391 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:39 smithi164 ceph-mon[32418]: Deploying daemon mds.e on smithi164 2024-04-25T13:10:39.392 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:39 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.72:6836/1560520314,v1:172.21.15.72:6837/1560520314] up:boot 2024-04-25T13:10:39.392 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:39 smithi164 ceph-mon[32418]: fsmap 4 up:standby 2024-04-25T13:10:39.392 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:39 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:10:39.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:39 smithi193 ceph-mon[32743]: Deploying daemon mds.e on smithi164 2024-04-25T13:10:39.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:39 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.72:6836/1560520314,v1:172.21.15.72:6837/1560520314] up:boot 2024-04-25T13:10:39.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:39 smithi193 ceph-mon[32743]: fsmap 4 up:standby 2024-04-25T13:10:39.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:39 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:10:39.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:39 smithi072 ceph-mon[29199]: Deploying daemon mds.e on smithi164 2024-04-25T13:10:39.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:39 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.72:6836/1560520314,v1:172.21.15.72:6837/1560520314] up:boot 2024-04-25T13:10:39.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:39 smithi072 ceph-mon[29199]: fsmap 4 up:standby 2024-04-25T13:10:39.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:39 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:10:39.613 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:39.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 -- 172.21.15.72:0/3715470183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc102040 msgr2=0x7fcccc1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:39.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 --2- 172.21.15.72:0/3715470183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc102040 0x7fcccc1024c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009900 tx=0x7fccc002a410 comp rx=0 tx=0).stop 2024-04-25T13:10:39.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 -- 172.21.15.72:0/3715470183 shutdown_connections 2024-04-25T13:10:39.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 --2- 172.21.15.72:0/3715470183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcccc102a00 0x7fcccc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 --2- 172.21.15.72:0/3715470183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc102040 0x7fcccc1024c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 --2- 172.21.15.72:0/3715470183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcccc100e60 0x7fcccc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 -- 172.21.15.72:0/3715470183 >> 172.21.15.72:0/3715470183 conn(0x7fcccc0fc440 msgr2=0x7fcccc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 -- 172.21.15.72:0/3715470183 shutdown_connections 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 -- 172.21.15.72:0/3715470183 wait complete. 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.725+0000 7fccd19e5640 1 Processor -- start 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fccd19e5640 1 -- start start 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fccd19e5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc100e60 0x7fcccc1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fccd19e5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcccc102040 0x7fcccc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fccd19e5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcccc102a00 0x7fcccc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fccd19e5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc10ca70 con 0x7fcccc100e60 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fccd19e5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc10c8f0 con 0x7fcccc102a00 2024-04-25T13:10:39.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fccd19e5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc10cbf0 con 0x7fcccc102040 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccaffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc100e60 0x7fcccc1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccaffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc100e60 0x7fcccc1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45424/0 (socket says 172.21.15.72:45424) 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccaffd640 1 -- 172.21.15.72:0/998446980 learned_addr learned my addr 172.21.15.72:0/998446980 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccb7fe640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcccc102a00 0x7fcccc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccaffd640 1 -- 172.21.15.72:0/998446980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcccc102040 msgr2=0x7fcccc198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fccca7fc640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcccc102040 0x7fcccc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccaffd640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcccc102040 0x7fcccc198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccaffd640 1 -- 172.21.15.72:0/998446980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcccc102a00 msgr2=0x7fcccc19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccaffd640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcccc102a00 0x7fcccc19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccaffd640 1 -- 172.21.15.72:0/998446980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc19fea0 con 0x7fcccc100e60 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccb7fe640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcccc102a00 0x7fcccc19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fccca7fc640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcccc102040 0x7fcccc198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:10:39.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.726+0000 7fcccaffd640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc100e60 0x7fcccc1982d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00b450 tx=0x7fccbc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:39.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.727+0000 7fccd09e3640 1 -- 172.21.15.72:0/998446980 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc015020 con 0x7fcccc100e60 2024-04-25T13:10:39.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.727+0000 7fccd09e3640 1 -- 172.21.15.72:0/998446980 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccbc011d40 con 0x7fcccc100e60 2024-04-25T13:10:39.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.727+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcccc1a0190 con 0x7fcccc100e60 2024-04-25T13:10:39.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.727+0000 7fccd09e3640 1 -- 172.21.15.72:0/998446980 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00f340 con 0x7fcccc100e60 2024-04-25T13:10:39.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.727+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcccc1a0710 con 0x7fcccc100e60 2024-04-25T13:10:39.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.728+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcccc1a09d0 con 0x7fcccc100e60 2024-04-25T13:10:39.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.729+0000 7fccd09e3640 1 -- 172.21.15.72:0/998446980 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fccbc018020 con 0x7fcccc100e60 2024-04-25T13:10:39.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.730+0000 7fccd09e3640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fccb40a3bb0 0x7fccb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:39.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.730+0000 7fccca7fc640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fccb40a3bb0 0x7fccb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:39.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.730+0000 7fccd09e3640 1 -- 172.21.15.72:0/998446980 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fccbc0cfac0 con 0x7fcccc100e60 2024-04-25T13:10:39.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.730+0000 7fccca7fc640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fccb40a3bb0 0x7fccb40a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fccc0006010 tx=0x7fccc0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:39.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.734+0000 7fccd09e3640 1 -- 172.21.15.72:0/998446980 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fccbc01f020 con 0x7fcccc100e60 2024-04-25T13:10:39.785 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:10:39 smithi164 systemd[1]: Starting Ceph mds.e for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:39.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.948+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcccc1a0ba0 con 0x7fcccc100e60 2024-04-25T13:10:39.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.948+0000 7fccd09e3640 1 -- 172.21.15.72:0/998446980 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7fccbc00d040 con 0x7fcccc100e60 2024-04-25T13:10:39.949 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:39.949 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:10:39.949 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 5 2024-04-25T13:10:39.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.950+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fccb40a3bb0 msgr2=0x7fccb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:39.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.950+0000 7fccd19e5640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fccb40a3bb0 0x7fccb40a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fccc0006010 tx=0x7fccc0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.950+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc100e60 msgr2=0x7fcccc1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.950+0000 7fccd19e5640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc100e60 0x7fcccc1982d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00b450 tx=0x7fccbc00b910 comp rx=0 tx=0).stop 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.951+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 shutdown_connections 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.951+0000 7fccd19e5640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fccb40a3bb0 0x7fccb40a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.951+0000 7fccd19e5640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcccc102a00 0x7fcccc19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.951+0000 7fccd19e5640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcccc102040 0x7fcccc198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.951+0000 7fccd19e5640 1 --2- 172.21.15.72:0/998446980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcccc100e60 0x7fcccc1982d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.951+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 >> 172.21.15.72:0/998446980 conn(0x7fcccc0fc440 msgr2=0x7fcccc06a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.951+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 shutdown_connections 2024-04-25T13:10:39.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:39.951+0000 7fccd19e5640 1 -- 172.21.15.72:0/998446980 wait complete. 2024-04-25T13:10:40.036 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:10:39 smithi164 podman[61605]: 2024-04-25 13:10:39.688663575 +0000 UTC m=+0.018575416 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:40.036 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:10:39 smithi164 podman[61605]: 2024-04-25 13:10:39.817848283 +0000 UTC m=+0.147760143 container create 857c802927013e9e87f8c091b132e634399336e7206af14dd217c89a1b997ccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:10:40.036 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:10:40 smithi164 podman[61605]: 2024-04-25 13:10:40.037111446 +0000 UTC m=+0.367023264 container init 857c802927013e9e87f8c091b132e634399336e7206af14dd217c89a1b997ccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T13:10:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:40 smithi072 ceph-mon[29199]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:40.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:40 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/998446980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:40.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:40 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:40.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:40 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:40.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:40 smithi164 ceph-mon[32418]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:40.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:40 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/998446980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:40.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:40 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:40.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:40 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:40.307 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:10:40 smithi164 podman[61605]: 2024-04-25 13:10:40.040108421 +0000 UTC m=+0.370020239 container start 857c802927013e9e87f8c091b132e634399336e7206af14dd217c89a1b997ccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T13:10:40.307 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:10:40 smithi164 ceph-mds[61624]: -- 172.21.15.164:0/4029673925 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bdf3f15a40 con 0x55bdf3ee8c00 2024-04-25T13:10:40.307 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:10:40 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e[61620]: starting mds.e at 2024-04-25T13:10:40.307 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:10:40 smithi164 bash[61605]: 857c802927013e9e87f8c091b132e634399336e7206af14dd217c89a1b997ccb 2024-04-25T13:10:40.307 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:10:40 smithi164 systemd[1]: Started Ceph mds.e for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:40.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:40 smithi193 ceph-mon[32743]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:40.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:40 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/998446980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:40.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:40 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:40.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:40 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:41.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:41 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.164:6834/3356589609,v1:172.21.15.164:6835/3356589609] up:boot 2024-04-25T13:10:41.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:41 smithi193 ceph-mon[32743]: fsmap 5 up:standby 2024-04-25T13:10:41.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:10:41.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:41.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:41.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:41.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:41 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:41.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:41 smithi193 ceph-mon[32743]: Deploying daemon mds.f on smithi193 2024-04-25T13:10:41.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:41 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.164:6834/3356589609,v1:172.21.15.164:6835/3356589609] up:boot 2024-04-25T13:10:41.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:41 smithi072 ceph-mon[29199]: fsmap 5 up:standby 2024-04-25T13:10:41.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:10:41.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:41.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:41.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:41.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:41 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:41.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:41 smithi072 ceph-mon[29199]: Deploying daemon mds.f on smithi193 2024-04-25T13:10:41.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:41 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.164:6834/3356589609,v1:172.21.15.164:6835/3356589609] up:boot 2024-04-25T13:10:41.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:41 smithi164 ceph-mon[32418]: fsmap 5 up:standby 2024-04-25T13:10:41.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:10:41.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:41.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:41.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:41.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:41 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:41.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:41 smithi164 ceph-mon[32418]: Deploying daemon mds.f on smithi193 2024-04-25T13:10:41.963 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:42.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.072+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2296481468 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc100e90 msgr2=0x7f81dc1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:42.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2296481468 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc100e90 0x7f81dc1012b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009900 tx=0x7f81d002a410 comp rx=0 tx=0).stop 2024-04-25T13:10:42.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2296481468 shutdown_connections 2024-04-25T13:10:42.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2296481468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81dc1029a0 0x7f81dc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:42.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2296481468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc101fe0 0x7f81dc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2296481468 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc100e90 0x7f81dc1012b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2296481468 >> 172.21.15.72:0/2296481468 conn(0x7f81dc0fc400 msgr2=0x7f81dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2296481468 shutdown_connections 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2296481468 wait complete. 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 Processor -- start 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 -- start start 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81dc100e90 0x7f81dc197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc101fe0 0x7f81dc197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc1029a0 0x7f81dc19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc10ca10 con 0x7f81dc1029a0 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc10c890 con 0x7f81dc101fe0 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e2c2a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc10cb90 con 0x7f81dc100e90 2024-04-25T13:10:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e11c7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc1029a0 0x7f81dc19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e11c7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc1029a0 0x7f81dc19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45450/0 (socket says 172.21.15.72:45450) 2024-04-25T13:10:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.073+0000 7f81e11c7640 1 -- 172.21.15.72:0/2215885834 learned_addr learned my addr 172.21.15.72:0/2215885834 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81dbfff640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc101fe0 0x7f81dc197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81e09c6640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81dc100e90 0x7f81dc197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81e11c7640 1 -- 172.21.15.72:0/2215885834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81dc100e90 msgr2=0x7f81dc197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81e11c7640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81dc100e90 0x7f81dc197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81e11c7640 1 -- 172.21.15.72:0/2215885834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc101fe0 msgr2=0x7f81dc197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81e11c7640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc101fe0 0x7f81dc197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81e11c7640 1 -- 172.21.15.72:0/2215885834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc19ed40 con 0x7f81dc1029a0 2024-04-25T13:10:42.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81e11c7640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc1029a0 0x7f81dc19e6d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009fd0 tx=0x7f81d4007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:42.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81d9ffb640 1 -- 172.21.15.72:0/2215885834 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d400de10 con 0x7f81dc1029a0 2024-04-25T13:10:42.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc069140 con 0x7f81dc1029a0 2024-04-25T13:10:42.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81d9ffb640 1 -- 172.21.15.72:0/2215885834 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d4003e20 con 0x7f81dc1029a0 2024-04-25T13:10:42.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81d9ffb640 1 -- 172.21.15.72:0/2215885834 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4004810 con 0x7f81dc1029a0 2024-04-25T13:10:42.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.074+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc069690 con 0x7f81dc1029a0 2024-04-25T13:10:42.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.075+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc068d70 con 0x7f81dc1029a0 2024-04-25T13:10:42.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.076+0000 7f81d9ffb640 1 -- 172.21.15.72:0/2215885834 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81d4003970 con 0x7f81dc1029a0 2024-04-25T13:10:42.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.077+0000 7f81d9ffb640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81cc0a3dd0 0x7f81cc0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:42.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.077+0000 7f81d9ffb640 1 -- 172.21.15.72:0/2215885834 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f81d40d15d0 con 0x7f81dc1029a0 2024-04-25T13:10:42.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.077+0000 7f81e09c6640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81cc0a3dd0 0x7f81cc0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:42.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.077+0000 7f81e09c6640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81cc0a3dd0 0x7f81cc0a6290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f81d0031500 tx=0x7f81d0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:42.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.081+0000 7f81d9ffb640 1 -- 172.21.15.72:0/2215885834 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81d409de00 con 0x7f81dc1029a0 2024-04-25T13:10:42.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:42 smithi072 ceph-mon[29199]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.299+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81dc107e20 con 0x7f81dc1029a0 2024-04-25T13:10:42.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.300+0000 7f81d9ffb640 1 -- 172.21.15.72:0/2215885834 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f81d40a2100 con 0x7f81dc1029a0 2024-04-25T13:10:42.300 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:42.300 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:10:42.301 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 6 2024-04-25T13:10:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81cc0a3dd0 msgr2=0x7f81cc0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81cc0a3dd0 0x7f81cc0a6290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f81d0031500 tx=0x7f81d0005a20 comp rx=0 tx=0).stop 2024-04-25T13:10:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc1029a0 msgr2=0x7f81dc19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc1029a0 0x7f81dc19e6d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009fd0 tx=0x7f81d4007590 comp rx=0 tx=0).stop 2024-04-25T13:10:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 shutdown_connections 2024-04-25T13:10:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81cc0a3dd0 0x7f81cc0a6290 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81dc1029a0 0x7f81dc19e6d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc101fe0 0x7f81dc197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 --2- 172.21.15.72:0/2215885834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81dc100e90 0x7f81dc197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 >> 172.21.15.72:0/2215885834 conn(0x7f81dc0fc400 msgr2=0x7f81dc1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 shutdown_connections 2024-04-25T13:10:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:42.302+0000 7f81e2c2a640 1 -- 172.21.15.72:0/2215885834 wait complete. 2024-04-25T13:10:42.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:42 smithi193 ceph-mon[32743]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:42.479 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:10:42 smithi193 systemd[1]: Starting Ceph mds.f for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:42.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:42 smithi164 ceph-mon[32418]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:42.731 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:10:42 smithi193 podman[62486]: 2024-04-25 13:10:42.480209391 +0000 UTC m=+0.098434303 container create 9aab990b038b1e5ba411bd8cf6776274110ee636ee05079ecafd2d7224ee4fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T13:10:42.731 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:10:42 smithi193 podman[62486]: 2024-04-25 13:10:42.399678491 +0000 UTC m=+0.017903426 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:42.731 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:10:42 smithi193 podman[62486]: 2024-04-25 13:10:42.682466454 +0000 UTC m=+0.300691366 container init 9aab990b038b1e5ba411bd8cf6776274110ee636ee05079ecafd2d7224ee4fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T13:10:42.731 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:10:42 smithi193 podman[62486]: 2024-04-25 13:10:42.68494236 +0000 UTC m=+0.303167272 container start 9aab990b038b1e5ba411bd8cf6776274110ee636ee05079ecafd2d7224ee4fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:10:42.731 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:10:42 smithi193 ceph-mds[62505]: -- 172.21.15.193:0/3501790874 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af45305a40 con 0x55af452d8800 2024-04-25T13:10:42.732 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:10:42 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f[62501]: starting mds.f at 2024-04-25T13:10:43.221 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:10:42 smithi193 bash[62486]: 9aab990b038b1e5ba411bd8cf6776274110ee636ee05079ecafd2d7224ee4fa1 2024-04-25T13:10:43.221 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:10:42 smithi193 systemd[1]: Started Ceph mds.f for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:43.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:43 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2215885834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:43.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:43.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:43.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:43.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:43.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:43.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:43 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:43.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:43 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2215885834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:43.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:43.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:43.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:43.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:43.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:43.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:43 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:43.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:43 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2215885834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:43.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:43.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:43.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:43.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:43.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:43.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:43 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:44.314 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:44.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.425+0000 7ff91ee7c640 1 -- 172.21.15.72:0/1785397523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff918107d10 msgr2=0x7ff91810a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:44.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.425+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/1785397523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff918107d10 0x7ff91810a100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff910009ca0 tx=0x7ff91002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 -- 172.21.15.72:0/1785397523 shutdown_connections 2024-04-25T13:10:44.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/1785397523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff918107d10 0x7ff91810a100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/1785397523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9181053e0 0x7ff9181077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/1785397523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102ab0 0x7ff918104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 -- 172.21.15.72:0/1785397523 >> 172.21.15.72:0/1785397523 conn(0x7ff9180fc400 msgr2=0x7ff9180fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 -- 172.21.15.72:0/1785397523 shutdown_connections 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 -- 172.21.15.72:0/1785397523 wait complete. 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 Processor -- start 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 -- start start 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102ab0 0x7ff9181981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9181053e0 0x7ff918198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff918107d10 0x7ff91819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff91810db40 con 0x7ff9181053e0 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff91810d9c0 con 0x7ff918107d10 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff91ee7c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff91810dcc0 con 0x7ff918102ab0 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff917fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9181053e0 0x7ff918198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.426+0000 7ff917fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9181053e0 0x7ff918198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45458/0 (socket says 172.21.15.72:45458) 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff917fff640 1 -- 172.21.15.72:0/73346743 learned_addr learned my addr 172.21.15.72:0/73346743 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff91cc18640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102ab0 0x7ff9181981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff91d419640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff918107d10 0x7ff91819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff917fff640 1 -- 172.21.15.72:0/73346743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102ab0 msgr2=0x7ff9181981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff917fff640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102ab0 0x7ff9181981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff917fff640 1 -- 172.21.15.72:0/73346743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff918107d10 msgr2=0x7ff91819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff917fff640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff918107d10 0x7ff91819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff917fff640 1 -- 172.21.15.72:0/73346743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91819fde0 con 0x7ff9181053e0 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff91cc18640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102ab0 0x7ff9181981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff91d419640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff918107d10 0x7ff91819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff917fff640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9181053e0 0x7ff918198700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c970 tx=0x7ff90c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff915ffb640 1 -- 172.21.15.72:0/73346743 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c007bb0 con 0x7ff9181053e0 2024-04-25T13:10:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff915ffb640 1 -- 172.21.15.72:0/73346743 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff90c007d50 con 0x7ff9181053e0 2024-04-25T13:10:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9181a00d0 con 0x7ff9181053e0 2024-04-25T13:10:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff915ffb640 1 -- 172.21.15.72:0/73346743 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00f450 con 0x7ff9181053e0 2024-04-25T13:10:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.427+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9181a0620 con 0x7ff9181053e0 2024-04-25T13:10:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.428+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9180fdef0 con 0x7ff9181053e0 2024-04-25T13:10:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.429+0000 7ff915ffb640 1 -- 172.21.15.72:0/73346743 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff90c004f20 con 0x7ff9181053e0 2024-04-25T13:10:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.430+0000 7ff915ffb640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9000a3bb0 0x7ff9000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.430+0000 7ff91cc18640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9000a3bb0 0x7ff9000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.430+0000 7ff915ffb640 1 -- 172.21.15.72:0/73346743 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff90c0d0030 con 0x7ff9181053e0 2024-04-25T13:10:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.431+0000 7ff91cc18640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9000a3bb0 0x7ff9000a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff9080058c0 tx=0x7ff908005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.435+0000 7ff915ffb640 1 -- 172.21.15.72:0/73346743 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff90c014030 con 0x7ff9181053e0 2024-04-25T13:10:44.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:44 smithi193 ceph-mon[32743]: Deploying daemon mds.c on smithi193 2024-04-25T13:10:44.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:44 smithi193 ceph-mon[32743]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:44.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:44 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.193:6834/1113925217,v1:172.21.15.193:6835/1113925217] up:boot 2024-04-25T13:10:44.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:44 smithi193 ceph-mon[32743]: fsmap 6 up:standby 2024-04-25T13:10:44.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:44 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:10:44.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:44 smithi072 ceph-mon[29199]: Deploying daemon mds.c on smithi193 2024-04-25T13:10:44.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:44 smithi072 ceph-mon[29199]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:44.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:44 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.193:6834/1113925217,v1:172.21.15.193:6835/1113925217] up:boot 2024-04-25T13:10:44.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:44 smithi072 ceph-mon[29199]: fsmap 6 up:standby 2024-04-25T13:10:44.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:44 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:10:44.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:44 smithi164 ceph-mon[32418]: Deploying daemon mds.c on smithi193 2024-04-25T13:10:44.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:44 smithi164 ceph-mon[32418]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:44.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:44 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.193:6834/1113925217,v1:172.21.15.193:6835/1113925217] up:boot 2024-04-25T13:10:44.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:44 smithi164 ceph-mon[32418]: fsmap 6 up:standby 2024-04-25T13:10:44.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:44 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:10:44.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.652+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff91802d770 con 0x7ff9181053e0 2024-04-25T13:10:44.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.652+0000 7ff915ffb640 1 -- 172.21.15.72:0/73346743 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7ff90c09ceb0 con 0x7ff9181053e0 2024-04-25T13:10:44.652 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:44.652 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T13:10:44.653 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 7 2024-04-25T13:10:44.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9000a3bb0 msgr2=0x7ff9000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9000a3bb0 0x7ff9000a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff9080058c0 tx=0x7ff908005850 comp rx=0 tx=0).stop 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9181053e0 msgr2=0x7ff918198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9181053e0 0x7ff918198700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c970 tx=0x7ff90c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 shutdown_connections 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9000a3bb0 0x7ff9000a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff918107d10 0x7ff91819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9181053e0 0x7ff918198700 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 --2- 172.21.15.72:0/73346743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102ab0 0x7ff9181981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 >> 172.21.15.72:0/73346743 conn(0x7ff9180fc400 msgr2=0x7ff9181034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 shutdown_connections 2024-04-25T13:10:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:44.654+0000 7ff91ee7c640 1 -- 172.21.15.72:0/73346743 wait complete. 2024-04-25T13:10:44.976 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:10:44 smithi193 systemd[1]: Starting Ceph mds.c for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:45.227 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:10:44 smithi193 podman[62802]: 2024-04-25 13:10:44.879922479 +0000 UTC m=+0.018028611 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:45.227 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:10:44 smithi193 podman[62802]: 2024-04-25 13:10:44.990839878 +0000 UTC m=+0.128945995 container create d985942e3e7cdd4f728d4c4b9cc9964ae8b4bebf996cfb2750514c845ca6941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:10:45.227 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:10:45 smithi193 podman[62802]: 2024-04-25 13:10:45.192997509 +0000 UTC m=+0.331103627 container init d985942e3e7cdd4f728d4c4b9cc9964ae8b4bebf996cfb2750514c845ca6941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T13:10:45.227 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:10:45 smithi193 podman[62802]: 2024-04-25 13:10:45.19572584 +0000 UTC m=+0.333831957 container start d985942e3e7cdd4f728d4c4b9cc9964ae8b4bebf996cfb2750514c845ca6941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T13:10:45.228 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:10:45 smithi193 ceph-mds[62821]: -- 172.21.15.193:0/3448554352 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d8612ba40 con 0x559d860fe800 2024-04-25T13:10:45.228 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:10:45 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c[62817]: starting mds.c at 2024-04-25T13:10:45.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:45 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/73346743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:45.481 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:10:45 smithi193 bash[62802]: d985942e3e7cdd4f728d4c4b9cc9964ae8b4bebf996cfb2750514c845ca6941f 2024-04-25T13:10:45.481 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:10:45 smithi193 systemd[1]: Started Ceph mds.c for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:45.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:45 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/73346743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:45.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:45 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/73346743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] up:boot 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: fsmap 7 up:standby 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:46.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:46 smithi164 ceph-mon[32418]: Deploying daemon mds.h on smithi164 2024-04-25T13:10:46.667 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:46.693 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:46.693 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] up:boot 2024-04-25T13:10:46.693 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: fsmap 7 up:standby 2024-04-25T13:10:46.693 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:10:46.694 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:46.694 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:46.694 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:46.694 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:46.694 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:46.694 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:46.694 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:46 smithi072 ceph-mon[29199]: Deploying daemon mds.h on smithi164 2024-04-25T13:10:46.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:46.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] up:boot 2024-04-25T13:10:46.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: fsmap 7 up:standby 2024-04-25T13:10:46.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:10:46.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:46.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:46.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:46.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:46.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:46.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:46.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:46 smithi193 ceph-mon[32743]: Deploying daemon mds.h on smithi164 2024-04-25T13:10:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.778+0000 7ff7f0908640 1 -- 172.21.15.72:0/1246127462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec1029a0 msgr2=0x7ff7ec1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.778+0000 7ff7f0908640 1 --2- 172.21.15.72:0/1246127462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec1029a0 0x7ff7ec1092c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009900 tx=0x7ff7dc02a410 comp rx=0 tx=0).stop 2024-04-25T13:10:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.779+0000 7ff7f0908640 1 -- 172.21.15.72:0/1246127462 shutdown_connections 2024-04-25T13:10:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.779+0000 7ff7f0908640 1 --2- 172.21.15.72:0/1246127462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec1029a0 0x7ff7ec1092c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.779+0000 7ff7f0908640 1 --2- 172.21.15.72:0/1246127462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7ec101fe0 0x7ff7ec102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.779+0000 7ff7f0908640 1 --2- 172.21.15.72:0/1246127462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec100e90 0x7ff7ec1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:46.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.779+0000 7ff7f0908640 1 -- 172.21.15.72:0/1246127462 >> 172.21.15.72:0/1246127462 conn(0x7ff7ec0fc400 msgr2=0x7ff7ec0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:46.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.779+0000 7ff7f0908640 1 -- 172.21.15.72:0/1246127462 shutdown_connections 2024-04-25T13:10:46.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.779+0000 7ff7f0908640 1 -- 172.21.15.72:0/1246127462 wait complete. 2024-04-25T13:10:46.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.779+0000 7ff7f0908640 1 Processor -- start 2024-04-25T13:10:46.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.779+0000 7ff7f0908640 1 -- start start 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7f0908640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec100e90 0x7ff7ec198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7f0908640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7ec101fe0 0x7ff7ec198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7f0908640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec1029a0 0x7ff7ec19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7f0908640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec10ca40 con 0x7ff7ec100e90 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7f0908640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec10c8c0 con 0x7ff7ec101fe0 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7f0908640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec10cbc0 con 0x7ff7ec1029a0 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ea59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec100e90 0x7ff7ec198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ea59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec100e90 0x7ff7ec198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44214/0 (socket says 172.21.15.72:44214) 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ea59c640 1 -- 172.21.15.72:0/2006687236 learned_addr learned my addr 172.21.15.72:0/2006687236 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7e9d9b640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7ec101fe0 0x7ff7ec198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ead9d640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec1029a0 0x7ff7ec19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ea59c640 1 -- 172.21.15.72:0/2006687236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec1029a0 msgr2=0x7ff7ec19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ea59c640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec1029a0 0x7ff7ec19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ea59c640 1 -- 172.21.15.72:0/2006687236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7ec101fe0 msgr2=0x7ff7ec198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ea59c640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7ec101fe0 0x7ff7ec198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ea59c640 1 -- 172.21.15.72:0/2006687236 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec19fe10 con 0x7ff7ec100e90 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7ea59c640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec100e90 0x7ff7ec198240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400ba50 tx=0x7ff7e400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7e37fe640 1 -- 172.21.15.72:0/2006687236 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4019070 con 0x7ff7ec100e90 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7e37fe640 1 -- 172.21.15.72:0/2006687236 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e400cbe0 con 0x7ff7ec100e90 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ec1a0100 con 0x7ff7ec100e90 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7e37fe640 1 -- 172.21.15.72:0/2006687236 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40075d0 con 0x7ff7ec100e90 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.780+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ec1a06d0 con 0x7ff7ec100e90 2024-04-25T13:10:46.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.781+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ec1a0990 con 0x7ff7ec100e90 2024-04-25T13:10:46.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.782+0000 7ff7e37fe640 1 -- 172.21.15.72:0/2006687236 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff7e401c070 con 0x7ff7ec100e90 2024-04-25T13:10:46.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.783+0000 7ff7e37fe640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff7d80a3bb0 0x7ff7d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:46.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.783+0000 7ff7e9d9b640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff7d80a3bb0 0x7ff7d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:46.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.783+0000 7ff7e37fe640 1 -- 172.21.15.72:0/2006687236 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff7e40cf530 con 0x7ff7ec100e90 2024-04-25T13:10:46.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.784+0000 7ff7e9d9b640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff7d80a3bb0 0x7ff7d80a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec199700 tx=0x7ff7d400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:46.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:46.788+0000 7ff7e37fe640 1 -- 172.21.15.72:0/2006687236 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7e409bd60 con 0x7ff7ec100e90 2024-04-25T13:10:47.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.004+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7ec1a0b60 con 0x7ff7ec100e90 2024-04-25T13:10:47.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.004+0000 7ff7e37fe640 1 -- 172.21.15.72:0/2006687236 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7ff7e40a0060 con 0x7ff7ec100e90 2024-04-25T13:10:47.004 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:47.004 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:10:47.004 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2024-04-25T13:10:47.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff7d80a3bb0 msgr2=0x7ff7d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:47.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff7d80a3bb0 0x7ff7d80a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec199700 tx=0x7ff7d400b390 comp rx=0 tx=0).stop 2024-04-25T13:10:47.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec100e90 msgr2=0x7ff7ec198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:47.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec100e90 0x7ff7ec198240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400ba50 tx=0x7ff7e400bf10 comp rx=0 tx=0).stop 2024-04-25T13:10:47.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 shutdown_connections 2024-04-25T13:10:47.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff7d80a3bb0 0x7ff7d80a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:47.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec1029a0 0x7ff7ec19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:47.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7ec101fe0 0x7ff7ec198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:47.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 --2- 172.21.15.72:0/2006687236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7ec100e90 0x7ff7ec198240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:47.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 >> 172.21.15.72:0/2006687236 conn(0x7ff7ec0fc400 msgr2=0x7ff7ec0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:47.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 shutdown_connections 2024-04-25T13:10:47.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:47.006+0000 7ff7f0908640 1 -- 172.21.15.72:0/2006687236 wait complete. 2024-04-25T13:10:47.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:47 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2006687236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:47.557 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:10:47 smithi164 systemd[1]: Starting Ceph mds.h for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:47.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:47 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2006687236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:47.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:47 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2006687236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:47.901 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:10:47 smithi164 podman[61915]: 2024-04-25 13:10:47.534020004 +0000 UTC m=+0.017784869 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:47.901 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:10:47 smithi164 podman[61915]: 2024-04-25 13:10:47.640388483 +0000 UTC m=+0.124153315 container create 9d9ba379a90a53476084f24bbd217392fb5ac298cefa48cbd8ff43cd05ac2f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:10:47.901 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:10:47 smithi164 podman[61915]: 2024-04-25 13:10:47.876604368 +0000 UTC m=+0.360369210 container init 9d9ba379a90a53476084f24bbd217392fb5ac298cefa48cbd8ff43cd05ac2f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T13:10:47.902 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:10:47 smithi164 podman[61915]: 2024-04-25 13:10:47.879215416 +0000 UTC m=+0.362980249 container start 9d9ba379a90a53476084f24bbd217392fb5ac298cefa48cbd8ff43cd05ac2f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T13:10:47.902 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:10:47 smithi164 ceph-mds[61935]: -- 172.21.15.164:0/167206156 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600bab99a40 con 0x5600bab6cc00 2024-04-25T13:10:48.292 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:10:47 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h[61931]: starting mds.h at 2024-04-25T13:10:48.292 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:10:47 smithi164 bash[61915]: 9d9ba379a90a53476084f24bbd217392fb5ac298cefa48cbd8ff43cd05ac2f0e 2024-04-25T13:10:48.293 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:10:47 smithi164 systemd[1]: Started Ceph mds.h for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:48 smithi164 ceph-mon[32418]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:48.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:48 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:48.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:48 smithi193 ceph-mon[32743]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:48.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:48.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:48.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:48.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:48.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:48.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:48 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:48.675 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:48 smithi072 ceph-mon[29199]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:48.675 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:48.675 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:48.675 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:48.675 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:48.675 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:48.675 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:48 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:49.018 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 -- 172.21.15.72:0/1032671957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4100e60 msgr2=0x7f8fc4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 --2- 172.21.15.72:0/1032671957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4100e60 0x7f8fc4101280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009900 tx=0x7f8fb802a410 comp rx=0 tx=0).stop 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 -- 172.21.15.72:0/1032671957 shutdown_connections 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 --2- 172.21.15.72:0/1032671957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4102a00 0x7f8fc4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 --2- 172.21.15.72:0/1032671957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4102040 0x7f8fc41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 --2- 172.21.15.72:0/1032671957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4100e60 0x7f8fc4101280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 -- 172.21.15.72:0/1032671957 >> 172.21.15.72:0/1032671957 conn(0x7f8fc40fc440 msgr2=0x7f8fc40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 -- 172.21.15.72:0/1032671957 shutdown_connections 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 -- 172.21.15.72:0/1032671957 wait complete. 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 Processor -- start 2024-04-25T13:10:49.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.121+0000 7f8fc967c640 1 -- start start 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc967c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4100e60 0x7f8fc4197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc967c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4102040 0x7f8fc41977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc967c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4102a00 0x7f8fc419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc967c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc410cad0 con 0x7f8fc4102a00 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc967c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc410c950 con 0x7f8fc4100e60 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc967c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc410cc50 con 0x7f8fc4102040 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4102a00 0x7f8fc419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4102a00 0x7f8fc419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44232/0 (socket says 172.21.15.72:44232) 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc37fe640 1 -- 172.21.15.72:0/4226014423 learned_addr learned my addr 172.21.15.72:0/4226014423 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc27fc640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4102040 0x7f8fc41977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc37fe640 1 -- 172.21.15.72:0/4226014423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4102040 msgr2=0x7f8fc41977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc2ffd640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4100e60 0x7f8fc4197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc37fe640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4102040 0x7f8fc41977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc37fe640 1 -- 172.21.15.72:0/4226014423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4100e60 msgr2=0x7f8fc4197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:49.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc37fe640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4100e60 0x7f8fc4197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc37fe640 1 -- 172.21.15.72:0/4226014423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc419eea0 con 0x7f8fc4102a00 2024-04-25T13:10:49.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc37fe640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4102a00 0x7f8fc419e830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00b970 tx=0x7f8fbc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:49.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fabfff640 1 -- 172.21.15.72:0/4226014423 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc019070 con 0x7f8fc4102a00 2024-04-25T13:10:49.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fabfff640 1 -- 172.21.15.72:0/4226014423 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fbc014440 con 0x7f8fc4102a00 2024-04-25T13:10:49.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc419f190 con 0x7f8fc4102a00 2024-04-25T13:10:49.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fabfff640 1 -- 172.21.15.72:0/4226014423 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0075d0 con 0x7f8fc4102a00 2024-04-25T13:10:49.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.122+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc419f710 con 0x7f8fc4102a00 2024-04-25T13:10:49.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.124+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc41082f0 con 0x7f8fc4102a00 2024-04-25T13:10:49.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.125+0000 7f8fabfff640 1 -- 172.21.15.72:0/4226014423 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fbc01c070 con 0x7f8fc4102a00 2024-04-25T13:10:49.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.125+0000 7f8fabfff640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8fac0a3bb0 0x7f8fac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:49.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.125+0000 7f8fabfff640 1 -- 172.21.15.72:0/4226014423 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f8fbc0cf590 con 0x7f8fc4102a00 2024-04-25T13:10:49.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.125+0000 7f8fc2ffd640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8fac0a3bb0 0x7f8fac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:49.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.126+0000 7f8fc2ffd640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8fac0a3bb0 0x7f8fac0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8fb802a930 tx=0x7f8fb8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:49.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.130+0000 7f8fabfff640 1 -- 172.21.15.72:0/4226014423 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fbc09c410 con 0x7f8fc4102a00 2024-04-25T13:10:49.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.352+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fc402d5c0 con 0x7f8fc4102a00 2024-04-25T13:10:49.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.353+0000 7f8fabfff640 1 -- 172.21.15.72:0/4226014423 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f8fbc09bd10 con 0x7f8fc4102a00 2024-04-25T13:10:49.353 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:49.353 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:10:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 9 2024-04-25T13:10:49.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.355+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8fac0a3bb0 msgr2=0x7f8fac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.355+0000 7f8fc967c640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8fac0a3bb0 0x7f8fac0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8fb802a930 tx=0x7f8fb8000bc0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.355+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4102a00 msgr2=0x7f8fc419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.355+0000 7f8fc967c640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4102a00 0x7f8fc419e830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00b970 tx=0x7f8fbc00be30 comp rx=0 tx=0).stop 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.356+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 shutdown_connections 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.356+0000 7f8fc967c640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8fac0a3bb0 0x7f8fac0a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.356+0000 7f8fc967c640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fc4102a00 0x7f8fc419e830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.356+0000 7f8fc967c640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4102040 0x7f8fc41977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.356+0000 7f8fc967c640 1 --2- 172.21.15.72:0/4226014423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4100e60 0x7f8fc4197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.356+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 >> 172.21.15.72:0/4226014423 conn(0x7f8fc40fc440 msgr2=0x7f8fc41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.356+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 shutdown_connections 2024-04-25T13:10:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:49.356+0000 7f8fc967c640 1 -- 172.21.15.72:0/4226014423 wait complete. 2024-04-25T13:10:49.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:49 smithi164 ceph-mon[32418]: Deploying daemon mds.i on smithi193 2024-04-25T13:10:49.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:49 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.164:6836/1335235896,v1:172.21.15.164:6837/1335235896] up:boot 2024-04-25T13:10:49.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:49 smithi164 ceph-mon[32418]: fsmap 8 up:standby 2024-04-25T13:10:49.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:49 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:10:49.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:49 smithi193 ceph-mon[32743]: Deploying daemon mds.i on smithi193 2024-04-25T13:10:49.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:49 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.164:6836/1335235896,v1:172.21.15.164:6837/1335235896] up:boot 2024-04-25T13:10:49.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:49 smithi193 ceph-mon[32743]: fsmap 8 up:standby 2024-04-25T13:10:49.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:49 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:10:49.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:49 smithi072 ceph-mon[29199]: Deploying daemon mds.i on smithi193 2024-04-25T13:10:49.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:49 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.164:6836/1335235896,v1:172.21.15.164:6837/1335235896] up:boot 2024-04-25T13:10:49.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:49 smithi072 ceph-mon[29199]: fsmap 8 up:standby 2024-04-25T13:10:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:49 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:10:50.231 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:10:50 smithi193 podman[63125]: 2024-04-25 13:10:50.06832818 +0000 UTC m=+0.018252358 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:50.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:50 smithi164 ceph-mon[32418]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:50.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:50 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/4226014423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:50 smithi193 ceph-mon[32743]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:50 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/4226014423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:50.731 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:10:50 smithi193 podman[63125]: 2024-04-25 13:10:50.379496915 +0000 UTC m=+0.329421059 container init ae363537a1acbe61378b2365269e38574c6ad377f2864e82378b1016127c55db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:10:50.731 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:10:50 smithi193 podman[63125]: 2024-04-25 13:10:50.382175981 +0000 UTC m=+0.332100128 container start ae363537a1acbe61378b2365269e38574c6ad377f2864e82378b1016127c55db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T13:10:50.731 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:10:50 smithi193 ceph-mds[63144]: -- 172.21.15.193:0/1392001289 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aeeb961a40 con 0x55aeeb935000 2024-04-25T13:10:50.731 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:10:50 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i[63140]: starting mds.i at 2024-04-25T13:10:50.731 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:10:50 smithi193 bash[63125]: ae363537a1acbe61378b2365269e38574c6ad377f2864e82378b1016127c55db 2024-04-25T13:10:50.731 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:10:50 smithi193 systemd[1]: Started Ceph mds.i for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:50.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:50 smithi072 ceph-mon[29199]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:50.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:50 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/4226014423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:51.368 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:51.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 -- 172.21.15.72:0/3574111348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc103190 msgr2=0x7f30dc103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:51.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 --2- 172.21.15.72:0/3574111348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc103190 0x7f30dc103610 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009900 tx=0x7f30d002a410 comp rx=0 tx=0).stop 2024-04-25T13:10:51.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 -- 172.21.15.72:0/3574111348 shutdown_connections 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 --2- 172.21.15.72:0/3574111348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30dc103b50 0x7f30dc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 --2- 172.21.15.72:0/3574111348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc103190 0x7f30dc103610 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 --2- 172.21.15.72:0/3574111348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30dc101fa0 0x7f30dc1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 -- 172.21.15.72:0/3574111348 >> 172.21.15.72:0/3574111348 conn(0x7f30dc0fd530 msgr2=0x7f30dc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 -- 172.21.15.72:0/3574111348 shutdown_connections 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 -- 172.21.15.72:0/3574111348 wait complete. 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 Processor -- start 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.468+0000 7f30e2336640 1 -- start start 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30e2336640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc101fa0 0x7f30dc198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30e2336640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30dc103190 0x7f30dc198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30e2336640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30dc103b50 0x7f30dc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30e2336640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc10db40 con 0x7f30dc101fa0 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30e2336640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc10d9c0 con 0x7f30dc103190 2024-04-25T13:10:51.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30e2336640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc10dcc0 con 0x7f30dc103b50 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30dbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc101fa0 0x7f30dc198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30dbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc101fa0 0x7f30dc198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44256/0 (socket says 172.21.15.72:44256) 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30dbfff640 1 -- 172.21.15.72:0/1238624282 learned_addr learned my addr 172.21.15.72:0/1238624282 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30db7fe640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30dc103190 0x7f30dc198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30e08d3640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30dc103b50 0x7f30dc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30dbfff640 1 -- 172.21.15.72:0/1238624282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30dc103b50 msgr2=0x7f30dc19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30dbfff640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30dc103b50 0x7f30dc19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30dbfff640 1 -- 172.21.15.72:0/1238624282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30dc103190 msgr2=0x7f30dc198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30dbfff640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30dc103190 0x7f30dc198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30dbfff640 1 -- 172.21.15.72:0/1238624282 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30dc19fe40 con 0x7f30dc101fa0 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30db7fe640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30dc103190 0x7f30dc198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.469+0000 7f30dbfff640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc101fa0 0x7f30dc198220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00b450 tx=0x7f30cc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.470+0000 7f30d97fa640 1 -- 172.21.15.72:0/1238624282 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc015020 con 0x7f30dc101fa0 2024-04-25T13:10:51.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.470+0000 7f30d97fa640 1 -- 172.21.15.72:0/1238624282 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30cc011d40 con 0x7f30dc101fa0 2024-04-25T13:10:51.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.470+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30dc1a0130 con 0x7f30dc101fa0 2024-04-25T13:10:51.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.470+0000 7f30d97fa640 1 -- 172.21.15.72:0/1238624282 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00f340 con 0x7f30dc101fa0 2024-04-25T13:10:51.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.470+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30dc1a0700 con 0x7f30dc101fa0 2024-04-25T13:10:51.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.471+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30dc1092e0 con 0x7f30dc101fa0 2024-04-25T13:10:51.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.472+0000 7f30d97fa640 1 -- 172.21.15.72:0/1238624282 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f30cc018020 con 0x7f30dc101fa0 2024-04-25T13:10:51.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.472+0000 7f30d97fa640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f30c40a3bb0 0x7f30c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:51.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.473+0000 7f30db7fe640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f30c40a3bb0 0x7f30c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:51.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.473+0000 7f30d97fa640 1 -- 172.21.15.72:0/1238624282 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f30cc0cfaa0 con 0x7f30dc101fa0 2024-04-25T13:10:51.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.473+0000 7f30db7fe640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f30c40a3bb0 0x7f30c40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30dc199730 tx=0x7f30d00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:51.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.477+0000 7f30d97fa640 1 -- 172.21.15.72:0/1238624282 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30cc01f020 con 0x7f30dc101fa0 2024-04-25T13:10:51.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.698+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30dc02d5c0 con 0x7f30dc101fa0 2024-04-25T13:10:51.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.699+0000 7f30d97fa640 1 -- 172.21.15.72:0/1238624282 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f30cc00d040 con 0x7f30dc101fa0 2024-04-25T13:10:51.699 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:51.699 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:10:51.699 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 10 2024-04-25T13:10:51.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f30c40a3bb0 msgr2=0x7f30c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:51.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f30c40a3bb0 0x7f30c40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30dc199730 tx=0x7f30d00314a0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc101fa0 msgr2=0x7f30dc198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:51.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc101fa0 0x7f30dc198220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00b450 tx=0x7f30cc00b910 comp rx=0 tx=0).stop 2024-04-25T13:10:51.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 shutdown_connections 2024-04-25T13:10:51.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f30c40a3bb0 0x7f30c40a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30dc103b50 0x7f30dc19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30dc103190 0x7f30dc198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 --2- 172.21.15.72:0/1238624282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30dc101fa0 0x7f30dc198220 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:51.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 >> 172.21.15.72:0/1238624282 conn(0x7f30dc0fd530 msgr2=0x7f30dc1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:51.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 shutdown_connections 2024-04-25T13:10:51.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:51.701+0000 7f30e2336640 1 -- 172.21.15.72:0/1238624282 wait complete. 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: Deploying daemon mds.g on smithi072 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] up:boot 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: fsmap 9 up:standby 2024-04-25T13:10:51.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:51 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:10:51.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:51.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:51.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:51.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:51.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:51.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:51.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: Deploying daemon mds.g on smithi072 2024-04-25T13:10:51.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:51.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] up:boot 2024-04-25T13:10:51.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: fsmap 9 up:standby 2024-04-25T13:10:51.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:51 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: Deploying daemon mds.g on smithi072 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] up:boot 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: fsmap 9 up:standby 2024-04-25T13:10:51.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:51 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:10:52.904 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:52 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1238624282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:52.904 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:10:52 smithi072 systemd[1]: Starting Ceph mds.g for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:52.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:52 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1238624282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:53.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:52 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1238624282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:53.241 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:10:53 smithi072 podman[70962]: 2024-04-25 13:10:53.119046748 +0000 UTC m=+0.112105715 container create 5f4ecb322965451dbe35b9fae744a315d49cc3209b7705fd6ea73ebce5c8f5b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:10:53.241 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:10:53 smithi072 podman[70962]: 2024-04-25 13:10:53.025018947 +0000 UTC m=+0.018077923 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:53.523 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:10:53 smithi072 podman[70962]: 2024-04-25 13:10:53.391877909 +0000 UTC m=+0.384936877 container init 5f4ecb322965451dbe35b9fae744a315d49cc3209b7705fd6ea73ebce5c8f5b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T13:10:53.524 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:10:53 smithi072 podman[70962]: 2024-04-25 13:10:53.394545408 +0000 UTC m=+0.387604381 container start 5f4ecb322965451dbe35b9fae744a315d49cc3209b7705fd6ea73ebce5c8f5b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:10:53.524 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:10:53 smithi072 ceph-mds[70981]: -- 172.21.15.72:0/737630302 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6988c3a40 con 0x55a698896800 2024-04-25T13:10:53.524 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:10:53 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-g[70977]: starting mds.g at 2024-04-25T13:10:53.524 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:10:53 smithi072 bash[70962]: 5f4ecb322965451dbe35b9fae744a315d49cc3209b7705fd6ea73ebce5c8f5b5 2024-04-25T13:10:53.524 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:10:53 smithi072 systemd[1]: Started Ceph mds.g for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:53.712 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:53.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.815+0000 7f266cf08640 1 -- 172.21.15.72:0/1593583078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f26681053a0 msgr2=0x7f2668107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:53.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.815+0000 7f266cf08640 1 --2- 172.21.15.72:0/1593583078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f26681053a0 0x7f2668107790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2658009900 tx=0x7f265802a410 comp rx=0 tx=0).stop 2024-04-25T13:10:53.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.815+0000 7f266cf08640 1 -- 172.21.15.72:0/1593583078 shutdown_connections 2024-04-25T13:10:53.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.815+0000 7f266cf08640 1 --2- 172.21.15.72:0/1593583078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2668107cd0 0x7f266810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:53.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.815+0000 7f266cf08640 1 --2- 172.21.15.72:0/1593583078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f26681053a0 0x7f2668107790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:53.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.815+0000 7f266cf08640 1 --2- 172.21.15.72:0/1593583078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2668102a70 0x7f2668104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:53.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.815+0000 7f266cf08640 1 -- 172.21.15.72:0/1593583078 >> 172.21.15.72:0/1593583078 conn(0x7f26680fc400 msgr2=0x7f26680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:53.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.815+0000 7f266cf08640 1 -- 172.21.15.72:0/1593583078 shutdown_connections 2024-04-25T13:10:53.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.815+0000 7f266cf08640 1 -- 172.21.15.72:0/1593583078 wait complete. 2024-04-25T13:10:53.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266cf08640 1 Processor -- start 2024-04-25T13:10:53.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266cf08640 1 -- start start 2024-04-25T13:10:53.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266cf08640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2668102a70 0x7f26681981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:53.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266cf08640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26681053a0 0x7f2668198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:53.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266cf08640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2668107cd0 0x7f266819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:53.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266cf08640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f266810dac0 con 0x7f2668107cd0 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266cf08640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f266810d940 con 0x7f26681053a0 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266cf08640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f266810dc40 con 0x7f2668102a70 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2666d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2668107cd0 0x7f266819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2666d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2668107cd0 0x7f266819f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44298/0 (socket says 172.21.15.72:44298) 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2666d9d640 1 -- 172.21.15.72:0/3356754964 learned_addr learned my addr 172.21.15.72:0/3356754964 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266659c640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2668102a70 0x7f26681981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2665d9b640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26681053a0 0x7f2668198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2666d9d640 1 -- 172.21.15.72:0/3356754964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2668102a70 msgr2=0x7f26681981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2666d9d640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2668102a70 0x7f26681981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2666d9d640 1 -- 172.21.15.72:0/3356754964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26681053a0 msgr2=0x7f2668198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2666d9d640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26681053a0 0x7f2668198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2666d9d640 1 -- 172.21.15.72:0/3356754964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266819fdf0 con 0x7f2668107cd0 2024-04-25T13:10:53.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f266659c640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2668102a70 0x7f26681981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:53.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.816+0000 7f2666d9d640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2668107cd0 0x7f266819f780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f266000cad0 tx=0x7f266000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:53.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.817+0000 7f26577fe640 1 -- 172.21.15.72:0/3356754964 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266000ae20 con 0x7f2668107cd0 2024-04-25T13:10:53.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.817+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26681a00e0 con 0x7f2668107cd0 2024-04-25T13:10:53.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.817+0000 7f26577fe640 1 -- 172.21.15.72:0/3356754964 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2660005020 con 0x7f2668107cd0 2024-04-25T13:10:53.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.817+0000 7f26577fe640 1 -- 172.21.15.72:0/3356754964 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266000f6b0 con 0x7f2668107cd0 2024-04-25T13:10:53.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.817+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26681a0650 con 0x7f2668107cd0 2024-04-25T13:10:53.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.818+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26680fdef0 con 0x7f2668107cd0 2024-04-25T13:10:53.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.819+0000 7f26577fe640 1 -- 172.21.15.72:0/3356754964 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26600051d0 con 0x7f2668107cd0 2024-04-25T13:10:53.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.819+0000 7f26577fe640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f26500a3bb0 0x7f26500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:53.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.819+0000 7f266659c640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f26500a3bb0 0x7f26500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:53.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.819+0000 7f26577fe640 1 -- 172.21.15.72:0/3356754964 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f26600d0090 con 0x7f2668107cd0 2024-04-25T13:10:53.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.820+0000 7f266659c640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f26500a3bb0 0x7f26500a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f265c00abe0 tx=0x7f265c00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:53.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:53.824+0000 7f26577fe640 1 -- 172.21.15.72:0/3356754964 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2660014030 con 0x7f2668107cd0 2024-04-25T13:10:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:53 smithi193 ceph-mon[32743]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:53.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:53 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:54.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:53 smithi072 ceph-mon[29199]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:54.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:54.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:54.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:54.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:54.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:54.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:53 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:54.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.048+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f266802d5c0 con 0x7f2668107cd0 2024-04-25T13:10:54.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.049+0000 7f26577fe640 1 -- 172.21.15.72:0/3356754964 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7f266009cf10 con 0x7f2668107cd0 2024-04-25T13:10:54.049 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:54.049 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:10:54.049 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 11 2024-04-25T13:10:54.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f26500a3bb0 msgr2=0x7f26500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:54.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f26500a3bb0 0x7f26500a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f265c00abe0 tx=0x7f265c00a5c0 comp rx=0 tx=0).stop 2024-04-25T13:10:54.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2668107cd0 msgr2=0x7f266819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:54.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2668107cd0 0x7f266819f780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f266000cad0 tx=0x7f266000cf90 comp rx=0 tx=0).stop 2024-04-25T13:10:54.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 shutdown_connections 2024-04-25T13:10:54.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f26500a3bb0 0x7f26500a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:54.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2668107cd0 0x7f266819f780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:54.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26681053a0 0x7f2668198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:54.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 --2- 172.21.15.72:0/3356754964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2668102a70 0x7f26681981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:54.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 >> 172.21.15.72:0/3356754964 conn(0x7f26680fc400 msgr2=0x7f26681033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:54.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 shutdown_connections 2024-04-25T13:10:54.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:54.051+0000 7f266cf08640 1 -- 172.21.15.72:0/3356754964 wait complete. 2024-04-25T13:10:54.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:53 smithi164 ceph-mon[32418]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:54.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:54.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:54.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:54.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:54.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:54.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:53 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:54.720 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:54 smithi072 ceph-mon[29199]: Deploying daemon mds.a on smithi072 2024-04-25T13:10:54.720 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:54 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.72:6838/3204730698,v1:172.21.15.72:6839/3204730698] up:boot 2024-04-25T13:10:54.720 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:54 smithi072 ceph-mon[29199]: fsmap 10 up:standby 2024-04-25T13:10:54.720 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:54 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:10:54.720 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:54 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3356754964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:54 smithi193 ceph-mon[32743]: Deploying daemon mds.a on smithi072 2024-04-25T13:10:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:54 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.72:6838/3204730698,v1:172.21.15.72:6839/3204730698] up:boot 2024-04-25T13:10:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:54 smithi193 ceph-mon[32743]: fsmap 10 up:standby 2024-04-25T13:10:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:54 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:10:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:54 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3356754964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:55.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:54 smithi164 ceph-mon[32418]: Deploying daemon mds.a on smithi072 2024-04-25T13:10:55.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:54 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.72:6838/3204730698,v1:172.21.15.72:6839/3204730698] up:boot 2024-04-25T13:10:55.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:54 smithi164 ceph-mon[32418]: fsmap 10 up:standby 2024-04-25T13:10:55.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:54 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:10:55.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:54 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3356754964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:55.600 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:10:55 smithi072 systemd[1]: Starting Ceph mds.a for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:55.932 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:55 smithi072 ceph-mon[29199]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:55.932 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:10:55 smithi072 podman[71307]: 2024-04-25 13:10:55.695852776 +0000 UTC m=+0.110139424 container create 1d514d7aff4a7def704c0748ffd393e1d0584c78d200b169bf091a91678f676c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T13:10:55.932 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:10:55 smithi072 podman[71307]: 2024-04-25 13:10:55.605393846 +0000 UTC m=+0.019680494 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:55.932 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:10:55 smithi072 podman[71307]: 2024-04-25 13:10:55.933313752 +0000 UTC m=+0.347600397 container init 1d514d7aff4a7def704c0748ffd393e1d0584c78d200b169bf091a91678f676c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:10:55.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:55 smithi193 ceph-mon[32743]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:56.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:55 smithi164 ceph-mon[32418]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:56.064 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:56.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.168+0000 7f78ad635640 1 -- 172.21.15.72:0/127237018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a8101fe0 msgr2=0x7f78a8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:56.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.168+0000 7f78ad635640 1 --2- 172.21.15.72:0/127237018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a8101fe0 0x7f78a8102460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7890009900 tx=0x7f789002a410 comp rx=0 tx=0).stop 2024-04-25T13:10:56.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.168+0000 7f78ad635640 1 -- 172.21.15.72:0/127237018 shutdown_connections 2024-04-25T13:10:56.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.168+0000 7f78ad635640 1 --2- 172.21.15.72:0/127237018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a81029a0 0x7f78a81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.168+0000 7f78ad635640 1 --2- 172.21.15.72:0/127237018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a8101fe0 0x7f78a8102460 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.168+0000 7f78ad635640 1 --2- 172.21.15.72:0/127237018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a8100e90 0x7f78a81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.168+0000 7f78ad635640 1 -- 172.21.15.72:0/127237018 >> 172.21.15.72:0/127237018 conn(0x7f78a80fc400 msgr2=0x7f78a80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:56.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.168+0000 7f78ad635640 1 -- 172.21.15.72:0/127237018 shutdown_connections 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.168+0000 7f78ad635640 1 -- 172.21.15.72:0/127237018 wait complete. 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78ad635640 1 Processor -- start 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78ad635640 1 -- start start 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78ad635640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a8100e90 0x7f78a8197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78ad635640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a8101fe0 0x7f78a81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78ad635640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a81029a0 0x7f78a819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78ad635640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f78a810c9d0 con 0x7f78a81029a0 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78ad635640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78a810c850 con 0x7f78a8101fe0 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78ad635640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78a810cb50 con 0x7f78a8100e90 2024-04-25T13:10:56.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78a77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a81029a0 0x7f78a819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78a77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a81029a0 0x7f78a819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42332/0 (socket says 172.21.15.72:42332) 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78a77fe640 1 -- 172.21.15.72:0/3060104115 learned_addr learned my addr 172.21.15.72:0/3060104115 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f789ffff640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a8101fe0 0x7f78a81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78a6ffd640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a8100e90 0x7f78a8197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78a77fe640 1 -- 172.21.15.72:0/3060104115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a8100e90 msgr2=0x7f78a8197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78a77fe640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a8100e90 0x7f78a8197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78a77fe640 1 -- 172.21.15.72:0/3060104115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a8101fe0 msgr2=0x7f78a81975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.169+0000 7f78a77fe640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a8101fe0 0x7f78a81975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.170+0000 7f78a77fe640 1 -- 172.21.15.72:0/3060104115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a819ec90 con 0x7f78a81029a0 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.170+0000 7f789ffff640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a8101fe0 0x7f78a81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.170+0000 7f78a77fe640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a81029a0 0x7f78a819e620 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f78a000eb10 tx=0x7f78a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:56.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.170+0000 7f78a4ff9640 1 -- 172.21.15.72:0/3060104115 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004b20 con 0x7f78a81029a0 2024-04-25T13:10:56.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.170+0000 7f78a4ff9640 1 -- 172.21.15.72:0/3060104115 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a0004cc0 con 0x7f78a81029a0 2024-04-25T13:10:56.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.170+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a819ef80 con 0x7f78a81029a0 2024-04-25T13:10:56.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.170+0000 7f78a4ff9640 1 -- 172.21.15.72:0/3060104115 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0010670 con 0x7f78a81029a0 2024-04-25T13:10:56.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.170+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a819f3f0 con 0x7f78a81029a0 2024-04-25T13:10:56.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.171+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a81081a0 con 0x7f78a81029a0 2024-04-25T13:10:56.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.172+0000 7f78a4ff9640 1 -- 172.21.15.72:0/3060104115 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f78a0005020 con 0x7f78a81029a0 2024-04-25T13:10:56.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.172+0000 7f78a4ff9640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f78940a3bb0 0x7f78940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:56.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.173+0000 7f78a4ff9640 1 -- 172.21.15.72:0/3060104115 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f78a00d0630 con 0x7f78a81029a0 2024-04-25T13:10:56.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.173+0000 7f78a6ffd640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f78940a3bb0 0x7f78940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:56.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.173+0000 7f78a6ffd640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f78940a3bb0 0x7f78940a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7898005e10 tx=0x7f7898005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:56.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.177+0000 7f78a4ff9640 1 -- 172.21.15.72:0/3060104115 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78a009d4b0 con 0x7f78a81029a0 2024-04-25T13:10:56.273 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:10:55 smithi072 podman[71307]: 2024-04-25 13:10:55.936146304 +0000 UTC m=+0.350432949 container start 1d514d7aff4a7def704c0748ffd393e1d0584c78d200b169bf091a91678f676c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-a, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:10:56.274 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:10:55 smithi072 ceph-mds[71328]: -- 172.21.15.72:0/2730225427 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f4cc23a40 con 0x560f4cbf6800 2024-04-25T13:10:56.274 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:10:55 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-a[71324]: starting mds.a at 2024-04-25T13:10:56.274 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:10:56 smithi072 bash[71307]: 1d514d7aff4a7def704c0748ffd393e1d0584c78d200b169bf091a91678f676c 2024-04-25T13:10:56.274 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:10:56 smithi072 systemd[1]: Started Ceph mds.a for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:56.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.394+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78a802d7c0 con 0x7f78a81029a0 2024-04-25T13:10:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.394+0000 7f78a4ff9640 1 -- 172.21.15.72:0/3060104115 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7f78a009cdb0 con 0x7f78a81029a0 2024-04-25T13:10:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:56.395 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:10:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 11 2024-04-25T13:10:56.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f78940a3bb0 msgr2=0x7f78940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:56.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f78940a3bb0 0x7f78940a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7898005e10 tx=0x7f7898005da0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a81029a0 msgr2=0x7f78a819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:56.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a81029a0 0x7f78a819e620 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f78a000eb10 tx=0x7f78a000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 shutdown_connections 2024-04-25T13:10:56.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f78940a3bb0 0x7f78940a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f78a81029a0 0x7f78a819e620 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a8101fe0 0x7f78a81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 --2- 172.21.15.72:0/3060104115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a8100e90 0x7f78a8197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:56.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 >> 172.21.15.72:0/3060104115 conn(0x7f78a80fc400 msgr2=0x7f78a8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:56.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 shutdown_connections 2024-04-25T13:10:56.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:56.396+0000 7f78ad635640 1 -- 172.21.15.72:0/3060104115 wait complete. 2024-04-25T13:10:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: Deploying daemon mds.b on smithi164 2024-04-25T13:10:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3060104115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:57.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.72:6840/882585461,v1:172.21.15.72:6841/882585461] up:boot 2024-04-25T13:10:57.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: fsmap 11 up:standby 2024-04-25T13:10:57.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:57 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:10:57.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: Deploying daemon mds.b on smithi164 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3060104115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.72:6840/882585461,v1:172.21.15.72:6841/882585461] up:boot 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: fsmap 11 up:standby 2024-04-25T13:10:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:57 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:10:57.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: Deploying daemon mds.b on smithi164 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3060104115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.72:6840/882585461,v1:172.21.15.72:6841/882585461] up:boot 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: fsmap 11 up:standby 2024-04-25T13:10:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:57 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:10:58.408 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:10:58.434 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:58 smithi072 ceph-mon[29199]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:58.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:58 smithi164 ceph-mon[32418]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:58.468 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:10:58 smithi164 systemd[1]: Starting Ceph mds.b for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:10:58.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:58 smithi193 ceph-mon[32743]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:10:58.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.519+0000 7f61eb42c640 1 -- 172.21.15.72:0/4179289809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4101fe0 msgr2=0x7f61e4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.519+0000 7f61eb42c640 1 --2- 172.21.15.72:0/4179289809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4101fe0 0x7f61e4102460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009900 tx=0x7f61d402a410 comp rx=0 tx=0).stop 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.519+0000 7f61eb42c640 1 -- 172.21.15.72:0/4179289809 shutdown_connections 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.519+0000 7f61eb42c640 1 --2- 172.21.15.72:0/4179289809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61e41029a0 0x7f61e41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.519+0000 7f61eb42c640 1 --2- 172.21.15.72:0/4179289809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4101fe0 0x7f61e4102460 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.519+0000 7f61eb42c640 1 --2- 172.21.15.72:0/4179289809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e4100e90 0x7f61e41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.519+0000 7f61eb42c640 1 -- 172.21.15.72:0/4179289809 >> 172.21.15.72:0/4179289809 conn(0x7f61e40fc440 msgr2=0x7f61e40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.519+0000 7f61eb42c640 1 -- 172.21.15.72:0/4179289809 shutdown_connections 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.519+0000 7f61eb42c640 1 -- 172.21.15.72:0/4179289809 wait complete. 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61eb42c640 1 Processor -- start 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61eb42c640 1 -- start start 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61eb42c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4100e90 0x7f61e4198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:58.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61eb42c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61e4101fe0 0x7f61e4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61eb42c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e41029a0 0x7f61e419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61eb42c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f61e410c9e0 con 0x7f61e4100e90 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61eb42c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61e410c860 con 0x7f61e41029a0 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61eb42c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f61e410cb60 con 0x7f61e4101fe0 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e91c8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4100e90 0x7f61e4198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e91c8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4100e90 0x7f61e4198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42356/0 (socket says 172.21.15.72:42356) 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e91c8640 1 -- 172.21.15.72:0/3502591574 learned_addr learned my addr 172.21.15.72:0/3502591574 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e89c7640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61e4101fe0 0x7f61e4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e91c8640 1 -- 172.21.15.72:0/3502591574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61e4101fe0 msgr2=0x7f61e4198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e91c8640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61e4101fe0 0x7f61e4198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e91c8640 1 -- 172.21.15.72:0/3502591574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e41029a0 msgr2=0x7f61e419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e99c9640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e41029a0 0x7f61e419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:58.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e91c8640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e41029a0 0x7f61e419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.520+0000 7f61e91c8640 1 -- 172.21.15.72:0/3502591574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e419fe30 con 0x7f61e4100e90 2024-04-25T13:10:58.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.521+0000 7f61e91c8640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4100e90 0x7f61e4198210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f61e000d950 tx=0x7f61e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:58.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.521+0000 7f61da7fc640 1 -- 172.21.15.72:0/3502591574 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0017030 con 0x7f61e4100e90 2024-04-25T13:10:58.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.521+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e41a0120 con 0x7f61e4100e90 2024-04-25T13:10:58.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.521+0000 7f61da7fc640 1 -- 172.21.15.72:0/3502591574 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61e0004020 con 0x7f61e4100e90 2024-04-25T13:10:58.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.521+0000 7f61da7fc640 1 -- 172.21.15.72:0/3502591574 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00047a0 con 0x7f61e4100e90 2024-04-25T13:10:58.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.521+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e41a0640 con 0x7f61e4100e90 2024-04-25T13:10:58.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.522+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e41a0900 con 0x7f61e4100e90 2024-04-25T13:10:58.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.523+0000 7f61da7fc640 1 -- 172.21.15.72:0/3502591574 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f61e0009a30 con 0x7f61e4100e90 2024-04-25T13:10:58.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.523+0000 7f61da7fc640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f61d00a3bb0 0x7f61d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:58.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.524+0000 7f61e89c7640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f61d00a3bb0 0x7f61d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:58.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.524+0000 7f61da7fc640 1 -- 172.21.15.72:0/3502591574 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f61e00d01d0 con 0x7f61e4100e90 2024-04-25T13:10:58.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.524+0000 7f61e89c7640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f61d00a3bb0 0x7f61d00a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f61d4000c00 tx=0x7f61d40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:58.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.527+0000 7f61da7fc640 1 -- 172.21.15.72:0/3502591574 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61e009cfb0 con 0x7f61e4100e90 2024-04-25T13:10:58.736 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:10:58 smithi164 podman[62241]: 2024-04-25 13:10:58.468448959 +0000 UTC m=+0.105574185 container create 32fe3eaa671778e827f4da3e299ee5eef9dc4772c2c3566503f264ca82a88acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T13:10:58.736 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:10:58 smithi164 podman[62241]: 2024-04-25 13:10:58.381077258 +0000 UTC m=+0.018202473 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:10:58.737 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:10:58 smithi164 podman[62241]: 2024-04-25 13:10:58.662525125 +0000 UTC m=+0.299650312 container init 32fe3eaa671778e827f4da3e299ee5eef9dc4772c2c3566503f264ca82a88acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:10:58.737 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:10:58 smithi164 podman[62241]: 2024-04-25 13:10:58.665110769 +0000 UTC m=+0.302235952 container start 32fe3eaa671778e827f4da3e299ee5eef9dc4772c2c3566503f264ca82a88acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:10:58.737 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:10:58 smithi164 ceph-mds[62260]: -- 172.21.15.164:0/3343903719 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d03295fa40 con 0x55d032932800 2024-04-25T13:10:58.737 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:10:58 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b[62256]: starting mds.b at 2024-04-25T13:10:58.737 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:10:58 smithi164 bash[62241]: 32fe3eaa671778e827f4da3e299ee5eef9dc4772c2c3566503f264ca82a88acf 2024-04-25T13:10:58.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.744+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61e41a0ad0 con 0x7f61e4100e90 2024-04-25T13:10:58.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.744+0000 7f61da7fc640 1 -- 172.21.15.72:0/3502591574 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9665 (secure 0 0 0) 0x7f61e009c8b0 con 0x7f61e4100e90 2024-04-25T13:10:58.743 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:10:58.743 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:10:58.744 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 12 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.746+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f61d00a3bb0 msgr2=0x7f61d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.746+0000 7f61eb42c640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f61d00a3bb0 0x7f61d00a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f61d4000c00 tx=0x7f61d40314a0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.746+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4100e90 msgr2=0x7f61e4198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.746+0000 7f61eb42c640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4100e90 0x7f61e4198210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f61e000d950 tx=0x7f61e000de10 comp rx=0 tx=0).stop 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.747+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 shutdown_connections 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.747+0000 7f61eb42c640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f61d00a3bb0 0x7f61d00a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.747+0000 7f61eb42c640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e41029a0 0x7f61e419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.747+0000 7f61eb42c640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61e4101fe0 0x7f61e4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.747+0000 7f61eb42c640 1 --2- 172.21.15.72:0/3502591574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61e4100e90 0x7f61e4198210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.747+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 >> 172.21.15.72:0/3502591574 conn(0x7f61e40fc440 msgr2=0x7f61e406a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.747+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 shutdown_connections 2024-04-25T13:10:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:10:58.747+0000 7f61eb42c640 1 -- 172.21.15.72:0/3502591574 wait complete. 2024-04-25T13:10:59.056 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:10:58 smithi164 systemd[1]: Started Ceph mds.b for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:10:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:59 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3502591574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:10:59 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:59.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:59 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3502591574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:59.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:10:59 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:59 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3502591574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:10:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:10:59.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:10:59 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:00 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] up:boot 2024-04-25T13:11:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:00 smithi193 ceph-mon[32743]: fsmap 12 up:standby 2024-04-25T13:11:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:00 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:11:00.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:00 smithi193 ceph-mon[32743]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:00.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:00 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] up:boot 2024-04-25T13:11:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:00 smithi072 ceph-mon[29199]: fsmap 12 up:standby 2024-04-25T13:11:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:00 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:11:00.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:00 smithi072 ceph-mon[29199]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:00 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] up:boot 2024-04-25T13:11:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:00 smithi164 ceph-mon[32418]: fsmap 12 up:standby 2024-04-25T13:11:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:00 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:11:00.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:00 smithi164 ceph-mon[32418]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:00.758 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:11:00.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 -- 172.21.15.72:0/2442124419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8103b50 msgr2=0x7f53f810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:00.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 --2- 172.21.15.72:0/2442124419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8103b50 0x7f53f810a3e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009ca0 tx=0x7f53f002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 -- 172.21.15.72:0/2442124419 shutdown_connections 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 --2- 172.21.15.72:0/2442124419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8103b50 0x7f53f810a3e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 --2- 172.21.15.72:0/2442124419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f8103190 0x7f53f8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 --2- 172.21.15.72:0/2442124419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f8101fa0 0x7f53f81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 -- 172.21.15.72:0/2442124419 >> 172.21.15.72:0/2442124419 conn(0x7f53f80fd530 msgr2=0x7f53f80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 -- 172.21.15.72:0/2442124419 shutdown_connections 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 -- 172.21.15.72:0/2442124419 wait complete. 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 Processor -- start 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.859+0000 7f53ff259640 1 -- start start 2024-04-25T13:11:00.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53ff259640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8101fa0 0x7f53f8075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53ff259640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f8103190 0x7f53f8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53ff259640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f8103b50 0x7f53f8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53ff259640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f53f810daf0 con 0x7f53f8101fa0 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53ff259640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53f810d970 con 0x7f53f8103b50 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53ff259640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53f810dc70 con 0x7f53f8103190 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fcff5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8101fa0 0x7f53f8075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fcff5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8101fa0 0x7f53f8075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42376/0 (socket says 172.21.15.72:42376) 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fcff5640 1 -- 172.21.15.72:0/4086232115 learned_addr learned my addr 172.21.15.72:0/4086232115 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53f7fff640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f8103190 0x7f53f8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fcff5640 1 -- 172.21.15.72:0/4086232115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f8103190 msgr2=0x7f53f8074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:00.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fcff5640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f8103190 0x7f53f8074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fd7f6640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f8103b50 0x7f53f8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fcff5640 1 -- 172.21.15.72:0/4086232115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f8103b50 msgr2=0x7f53f8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fcff5640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f8103b50 0x7f53f8074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fcff5640 1 -- 172.21.15.72:0/4086232115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f8075110 con 0x7f53f8101fa0 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53fcff5640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8101fa0 0x7f53f8075d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f53e800d950 tx=0x7f53e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53f5ffb640 1 -- 172.21.15.72:0/4086232115 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8017020 con 0x7f53f8101fa0 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53f5ffb640 1 -- 172.21.15.72:0/4086232115 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e8009c30 con 0x7f53f8101fa0 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f81a2260 con 0x7f53f8101fa0 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53f5ffb640 1 -- 172.21.15.72:0/4086232115 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e800f6b0 con 0x7f53f8101fa0 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.860+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f81a27b0 con 0x7f53f8101fa0 2024-04-25T13:11:00.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.861+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f81092c0 con 0x7f53f8101fa0 2024-04-25T13:11:00.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.862+0000 7f53f5ffb640 1 -- 172.21.15.72:0/4086232115 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f53e801a020 con 0x7f53f8101fa0 2024-04-25T13:11:00.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.863+0000 7f53f5ffb640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f53e00a3bb0 0x7f53e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:00.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.863+0000 7f53f7fff640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f53e00a3bb0 0x7f53e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:00.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.863+0000 7f53f5ffb640 1 -- 172.21.15.72:0/4086232115 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f53e80cfd10 con 0x7f53f8101fa0 2024-04-25T13:11:00.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.864+0000 7f53f7fff640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f53e00a3bb0 0x7f53e00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53f8075a70 tx=0x7f53ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:00.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:00.868+0000 7f53f5ffb640 1 -- 172.21.15.72:0/4086232115 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53e8009de0 con 0x7f53f8101fa0 2024-04-25T13:11:01.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.082+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53f802d5c0 con 0x7f53f8101fa0 2024-04-25T13:11:01.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.083+0000 7f53f5ffb640 1 -- 172.21.15.72:0/4086232115 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7f53e809cb90 con 0x7f53f8101fa0 2024-04-25T13:11:01.082 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:01.082 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1444196308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1444196308},{"type":"v1","addr":"172.21.15.164:6839","nonce":1444196308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T13:11:01.082 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 13 2024-04-25T13:11:01.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f53e00a3bb0 msgr2=0x7f53e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f53e00a3bb0 0x7f53e00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53f8075a70 tx=0x7f53ec005f90 comp rx=0 tx=0).stop 2024-04-25T13:11:01.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8101fa0 msgr2=0x7f53f8075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8101fa0 0x7f53f8075d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f53e800d950 tx=0x7f53e800de10 comp rx=0 tx=0).stop 2024-04-25T13:11:01.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 shutdown_connections 2024-04-25T13:11:01.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f53e00a3bb0 0x7f53e00a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f8103b50 0x7f53f8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f8103190 0x7f53f8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 --2- 172.21.15.72:0/4086232115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53f8101fa0 0x7f53f8075d00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 >> 172.21.15.72:0/4086232115 conn(0x7f53f80fd530 msgr2=0x7f53f8106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:01.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 shutdown_connections 2024-04-25T13:11:01.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.085+0000 7f53ff259640 1 -- 172.21.15.72:0/4086232115 wait complete. 2024-04-25T13:11:01.095 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:11:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 -- 172.21.15.72:0/895912349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a0103b70 msgr2=0x7f55a010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 --2- 172.21.15.72:0/895912349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a0103b70 0x7f55a010a400 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5598009ca0 tx=0x7f559802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 -- 172.21.15.72:0/895912349 shutdown_connections 2024-04-25T13:11:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 --2- 172.21.15.72:0/895912349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a0103b70 0x7f55a010a400 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 --2- 172.21.15.72:0/895912349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a01031b0 0x7f55a0103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 --2- 172.21.15.72:0/895912349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0101fc0 0x7f55a01023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 -- 172.21.15.72:0/895912349 >> 172.21.15.72:0/895912349 conn(0x7f55a00fd530 msgr2=0x7f55a00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 -- 172.21.15.72:0/895912349 shutdown_connections 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 -- 172.21.15.72:0/895912349 wait complete. 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.199+0000 7f55a6d75640 1 Processor -- start 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a6d75640 1 -- start start 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a6d75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0101fc0 0x7f55a0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a6d75640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01031b0 0x7f55a0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a6d75640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a0103b70 0x7f55a0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a6d75640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f55a010db00 con 0x7f55a01031b0 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a6d75640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55a010d980 con 0x7f55a0103b70 2024-04-25T13:11:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a6d75640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55a010dc80 con 0x7f55a0101fc0 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01031b0 0x7f55a0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01031b0 0x7f55a0074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42386/0 (socket says 172.21.15.72:42386) 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559ffff640 1 -- 172.21.15.72:0/3941527570 learned_addr learned my addr 172.21.15.72:0/3941527570 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a4b11640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0101fc0 0x7f55a0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a5312640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a0103b70 0x7f55a0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559ffff640 1 -- 172.21.15.72:0/3941527570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0101fc0 msgr2=0x7f55a0075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559ffff640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0101fc0 0x7f55a0075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559ffff640 1 -- 172.21.15.72:0/3941527570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a0103b70 msgr2=0x7f55a0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559ffff640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a0103b70 0x7f55a0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559ffff640 1 -- 172.21.15.72:0/3941527570 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a0075110 con 0x7f55a01031b0 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559ffff640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01031b0 0x7f55a0074340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f558c00e9f0 tx=0x7f558c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559dffb640 1 -- 172.21.15.72:0/3941527570 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c00cd80 con 0x7f55a01031b0 2024-04-25T13:11:01.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a01a2290 con 0x7f55a01031b0 2024-04-25T13:11:01.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559dffb640 1 -- 172.21.15.72:0/3941527570 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f558c0108e0 con 0x7f55a01031b0 2024-04-25T13:11:01.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.200+0000 7f559dffb640 1 -- 172.21.15.72:0/3941527570 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c018980 con 0x7f55a01031b0 2024-04-25T13:11:01.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.201+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a01a27e0 con 0x7f55a01031b0 2024-04-25T13:11:01.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.202+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a01093b0 con 0x7f55a01031b0 2024-04-25T13:11:01.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.203+0000 7f559dffb640 1 -- 172.21.15.72:0/3941527570 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f558c010430 con 0x7f55a01031b0 2024-04-25T13:11:01.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.203+0000 7f559dffb640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f55900a3bb0 0x7f55900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.203+0000 7f55a4b11640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f55900a3bb0 0x7f55900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.203+0000 7f559dffb640 1 -- 172.21.15.72:0/3941527570 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f558c0d0250 con 0x7f55a01031b0 2024-04-25T13:11:01.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.204+0000 7f55a4b11640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f55900a3bb0 0x7f55900a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5594005fd0 tx=0x7f559400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:01.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.208+0000 7f559dffb640 1 -- 172.21.15.72:0/3941527570 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f558c09ca80 con 0x7f55a01031b0 2024-04-25T13:11:01.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.393+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55a002d770 con 0x7f55a01031b0 2024-04-25T13:11:01.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.394+0000 7f559dffb640 1 -- 172.21.15.72:0/3941527570 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18480 (secure 0 0 0) 0x7f558c0a0d80 con 0x7f55a01031b0 2024-04-25T13:11:01.395 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:01.395 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":84,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:10:15.483925+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:11:01.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.396+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f55900a3bb0 msgr2=0x7f55900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.396+0000 7f55a6d75640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f55900a3bb0 0x7f55900a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5594005fd0 tx=0x7f559400b390 comp rx=0 tx=0).stop 2024-04-25T13:11:01.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01031b0 msgr2=0x7f55a0074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01031b0 0x7f55a0074340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f558c00e9f0 tx=0x7f558c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 shutdown_connections 2024-04-25T13:11:01.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f55900a3bb0 0x7f55900a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a0103b70 0x7f55a0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01031b0 0x7f55a0074340 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 --2- 172.21.15.72:0/3941527570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0101fc0 0x7f55a0075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 >> 172.21.15.72:0/3941527570 conn(0x7f55a00fd530 msgr2=0x7f55a0106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:01.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 shutdown_connections 2024-04-25T13:11:01.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.397+0000 7f55a6d75640 1 -- 172.21.15.72:0/3941527570 wait complete. 2024-04-25T13:11:01.409 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:06:33.150709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:11:01.409 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:11:01.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.512+0000 7fd4755f7640 1 -- 172.21.15.72:0/3375988832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 msgr2=0x7fd47010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.512+0000 7fd4755f7640 1 --2- 172.21.15.72:0/3375988832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 0x7fd47010a3e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd468009ca0 tx=0x7fd46802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.512+0000 7fd4755f7640 1 -- 172.21.15.72:0/3375988832 shutdown_connections 2024-04-25T13:11:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.512+0000 7fd4755f7640 1 --2- 172.21.15.72:0/3375988832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 0x7fd47010a3e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.512+0000 7fd4755f7640 1 --2- 172.21.15.72:0/3375988832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470103190 0x7fd470103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.512+0000 7fd4755f7640 1 --2- 172.21.15.72:0/3375988832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd470101fa0 0x7fd4701023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.512+0000 7fd4755f7640 1 -- 172.21.15.72:0/3375988832 >> 172.21.15.72:0/3375988832 conn(0x7fd4700fd530 msgr2=0x7fd4700ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.512+0000 7fd4755f7640 1 -- 172.21.15.72:0/3375988832 shutdown_connections 2024-04-25T13:11:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.512+0000 7fd4755f7640 1 -- 172.21.15.72:0/3375988832 wait complete. 2024-04-25T13:11:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.513+0000 7fd4755f7640 1 Processor -- start 2024-04-25T13:11:01.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.513+0000 7fd4755f7640 1 -- start start 2024-04-25T13:11:01.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd4755f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd470101fa0 0x7fd4701981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd4755f7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470103190 0x7fd4701986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd4755f7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 0x7fd47019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd4755f7640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd47010dae0 con 0x7fd470103b50 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd4755f7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd47010d960 con 0x7fd470101fa0 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd4755f7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd47010dc60 con 0x7fd470103190 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 0x7fd47019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 0x7fd47019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42410/0 (socket says 172.21.15.72:42410) 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46f7fe640 1 -- 172.21.15.72:0/2378923579 learned_addr learned my addr 172.21.15.72:0/2378923579 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46effd640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd470101fa0 0x7fd4701981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46e7fc640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470103190 0x7fd4701986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46f7fe640 1 -- 172.21.15.72:0/2378923579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470103190 msgr2=0x7fd4701986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46f7fe640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470103190 0x7fd4701986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46f7fe640 1 -- 172.21.15.72:0/2378923579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd470101fa0 msgr2=0x7fd4701981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46f7fe640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd470101fa0 0x7fd4701981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46f7fe640 1 -- 172.21.15.72:0/2378923579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47019fdc0 con 0x7fd470103b50 2024-04-25T13:11:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd46f7fe640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 0x7fd47019f750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd4680097b0 tx=0x7fd468031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd457fff640 1 -- 172.21.15.72:0/2378923579 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468003bf0 con 0x7fd470103b50 2024-04-25T13:11:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4701a0050 con 0x7fd470103b50 2024-04-25T13:11:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd457fff640 1 -- 172.21.15.72:0/2378923579 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd468003d90 con 0x7fd470103b50 2024-04-25T13:11:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd457fff640 1 -- 172.21.15.72:0/2378923579 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46802fc70 con 0x7fd470103b50 2024-04-25T13:11:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.514+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4701a05a0 con 0x7fd470103b50 2024-04-25T13:11:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.515+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd470109340 con 0x7fd470103b50 2024-04-25T13:11:01.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.516+0000 7fd457fff640 1 -- 172.21.15.72:0/2378923579 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd468004040 con 0x7fd470103b50 2024-04-25T13:11:01.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.517+0000 7fd457fff640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4580a3bb0 0x7fd4580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.517+0000 7fd46effd640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4580a3bb0 0x7fd4580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.517+0000 7fd457fff640 1 -- 172.21.15.72:0/2378923579 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd4680f1750 con 0x7fd470103b50 2024-04-25T13:11:01.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.518+0000 7fd46effd640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4580a3bb0 0x7fd4580a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd464005fd0 tx=0x7fd46400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:01.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.521+0000 7fd457fff640 1 -- 172.21.15.72:0/2378923579 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4680be530 con 0x7fd470103b50 2024-04-25T13:11:01.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.710+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd47002d5c0 con 0x7fd470103b50 2024-04-25T13:11:01.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.711+0000 7fd457fff640 1 -- 172.21.15.72:0/2378923579 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fd4680bde30 con 0x7fd470103b50 2024-04-25T13:11:01.711 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-25T13:11:01.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4580a3bb0 msgr2=0x7fd4580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4580a3bb0 0x7fd4580a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd464005fd0 tx=0x7fd46400b410 comp rx=0 tx=0).stop 2024-04-25T13:11:01.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 msgr2=0x7fd47019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 0x7fd47019f750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd4680097b0 tx=0x7fd468031840 comp rx=0 tx=0).stop 2024-04-25T13:11:01.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 shutdown_connections 2024-04-25T13:11:01.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4580a3bb0 0x7fd4580a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470103b50 0x7fd47019f750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470103190 0x7fd4701986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 --2- 172.21.15.72:0/2378923579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd470101fa0 0x7fd4701981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 >> 172.21.15.72:0/2378923579 conn(0x7fd4700fd530 msgr2=0x7fd470106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:01.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 shutdown_connections 2024-04-25T13:11:01.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.714+0000 7fd4755f7640 1 -- 172.21.15.72:0/2378923579 wait complete. 2024-04-25T13:11:01.728 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.728 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.728 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.729 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.729 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.729 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.729 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/4086232115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:11:01.729 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:01.729 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:01.729 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:01.729 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.730 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:01.730 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:01 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3941527570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:01.731 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:11:01.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.842+0000 7f11a01a2640 1 -- 172.21.15.72:0/228793299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198103b70 msgr2=0x7f119810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.842+0000 7f11a01a2640 1 --2- 172.21.15.72:0/228793299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198103b70 0x7f119810a400 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1190009ca0 tx=0x7f119002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.842+0000 7f11a01a2640 1 -- 172.21.15.72:0/228793299 shutdown_connections 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.842+0000 7f11a01a2640 1 --2- 172.21.15.72:0/228793299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198103b70 0x7f119810a400 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.842+0000 7f11a01a2640 1 --2- 172.21.15.72:0/228793299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11981031b0 0x7f1198103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.842+0000 7f11a01a2640 1 --2- 172.21.15.72:0/228793299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1198101fc0 0x7f11981023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.842+0000 7f11a01a2640 1 -- 172.21.15.72:0/228793299 >> 172.21.15.72:0/228793299 conn(0x7f11980fd530 msgr2=0x7f11980ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 -- 172.21.15.72:0/228793299 shutdown_connections 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 -- 172.21.15.72:0/228793299 wait complete. 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 Processor -- start 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 -- start start 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198101fc0 0x7f11981981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11981031b0 0x7f11981986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1198103b70 0x7f119819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f119810dac0 con 0x7f1198101fc0 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f119810d940 con 0x7f1198103b70 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f11a01a2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f119810dc40 con 0x7f11981031b0 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f119df3e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198101fc0 0x7f11981981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f119df3e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198101fc0 0x7f11981981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42422/0 (socket says 172.21.15.72:42422) 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.843+0000 7f119df3e640 1 -- 172.21.15.72:0/4022940477 learned_addr learned my addr 172.21.15.72:0/4022940477 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f119e73f640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1198103b70 0x7f119819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f119d73d640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11981031b0 0x7f11981986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f119df3e640 1 -- 172.21.15.72:0/4022940477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11981031b0 msgr2=0x7f11981986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f119df3e640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11981031b0 0x7f11981986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f119df3e640 1 -- 172.21.15.72:0/4022940477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1198103b70 msgr2=0x7f119819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:01.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f119df3e640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1198103b70 0x7f119819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:01.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f119df3e640 1 -- 172.21.15.72:0/4022940477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119819fdd0 con 0x7f1198101fc0 2024-04-25T13:11:01.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f119df3e640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198101fc0 0x7f11981981b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f118800d770 tx=0x7f118800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:01.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f118effd640 1 -- 172.21.15.72:0/4022940477 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188017070 con 0x7f1198101fc0 2024-04-25T13:11:01.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f118effd640 1 -- 172.21.15.72:0/4022940477 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1188009c30 con 0x7f1198101fc0 2024-04-25T13:11:01.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11981a00c0 con 0x7f1198101fc0 2024-04-25T13:11:01.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f118effd640 1 -- 172.21.15.72:0/4022940477 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188003cc0 con 0x7f1198101fc0 2024-04-25T13:11:01.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.844+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11981a0610 con 0x7f1198101fc0 2024-04-25T13:11:01.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.845+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11981092e0 con 0x7f1198101fc0 2024-04-25T13:11:01.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.847+0000 7f118effd640 1 -- 172.21.15.72:0/4022940477 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1188003e60 con 0x7f1198101fc0 2024-04-25T13:11:01.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.847+0000 7f118effd640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f11840a3bb0 0x7f11840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:01.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.848+0000 7f119d73d640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f11840a3bb0 0x7f11840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:01.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.848+0000 7f118effd640 1 -- 172.21.15.72:0/4022940477 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f11880cfca0 con 0x7f1198101fc0 2024-04-25T13:11:01.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.848+0000 7f119d73d640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f11840a3bb0 0x7f11840a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11981996c0 tx=0x7f1194005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:01.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:01.851+0000 7f118effd640 1 -- 172.21.15.72:0/4022940477 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1188009de0 con 0x7f1198101fc0 2024-04-25T13:11:01.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/4086232115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:11:01.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:01.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:01.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:01.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:01.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:01.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:01 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3941527570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:02.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.036+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1198062b80 con 0x7f1198101fc0 2024-04-25T13:11:02.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.036+0000 7f118effd640 1 -- 172.21.15.72:0/4022940477 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7f118809c4b0 con 0x7f1198101fc0 2024-04-25T13:11:02.036 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:02.036 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2024-04-25T13:11:02.037 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:11:02.037 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 1, 2024-04-25T13:11:02.037 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": ".mgr" 2024-04-25T13:11:02.037 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:11:02.037 INFO:teuthology.orchestra.run.smithi072.stdout:] 2024-04-25T13:11:02.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f11840a3bb0 msgr2=0x7f11840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f11840a3bb0 0x7f11840a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11981996c0 tx=0x7f1194005f90 comp rx=0 tx=0).stop 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198101fc0 msgr2=0x7f11981981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198101fc0 0x7f11981981b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f118800d770 tx=0x7f118800dc30 comp rx=0 tx=0).stop 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 shutdown_connections 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f11840a3bb0 0x7f11840a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1198103b70 0x7f119819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11981031b0 0x7f11981986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 --2- 172.21.15.72:0/4022940477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1198101fc0 0x7f11981981b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.038+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 >> 172.21.15.72:0/4022940477 conn(0x7f11980fd530 msgr2=0x7f11981064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.039+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 shutdown_connections 2024-04-25T13:11:02.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.039+0000 7f11a01a2640 1 -- 172.21.15.72:0/4022940477 wait complete. 2024-04-25T13:11:02.043 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:02.043 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:02.043 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/4086232115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:02.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:01 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3941527570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:02.049 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:11:02.049 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:11:02.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.155+0000 7faf0f4e1640 1 -- 172.21.15.72:0/2954717976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 msgr2=0x7faf08102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:02.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.155+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/2954717976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 0x7faf08102460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faef8009900 tx=0x7faef802a410 comp rx=0 tx=0).stop 2024-04-25T13:11:02.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.155+0000 7faf0f4e1640 1 -- 172.21.15.72:0/2954717976 shutdown_connections 2024-04-25T13:11:02.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.155+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/2954717976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf081029a0 0x7faf081092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.155+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/2954717976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 0x7faf08102460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.155+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/2954717976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf08100e90 0x7faf081012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.155+0000 7faf0f4e1640 1 -- 172.21.15.72:0/2954717976 >> 172.21.15.72:0/2954717976 conn(0x7faf080fc400 msgr2=0x7faf080fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.155+0000 7faf0f4e1640 1 -- 172.21.15.72:0/2954717976 shutdown_connections 2024-04-25T13:11:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.155+0000 7faf0f4e1640 1 -- 172.21.15.72:0/2954717976 wait complete. 2024-04-25T13:11:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0f4e1640 1 Processor -- start 2024-04-25T13:11:02.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0f4e1640 1 -- start start 2024-04-25T13:11:02.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0f4e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf08100e90 0x7faf08069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:02.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0f4e1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 0x7faf0819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:02.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0f4e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf081029a0 0x7faf0819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:02.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0f4e1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faf0810c9f0 con 0x7faf08101fe0 2024-04-25T13:11:02.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0f4e1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf0810c870 con 0x7faf081029a0 2024-04-25T13:11:02.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0f4e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf0810cb70 con 0x7faf08100e90 2024-04-25T13:11:02.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0ca7c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 0x7faf0819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:02.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0ca7c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 0x7faf0819cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42436/0 (socket says 172.21.15.72:42436) 2024-04-25T13:11:02.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0ca7c640 1 -- 172.21.15.72:0/1588549094 learned_addr learned my addr 172.21.15.72:0/1588549094 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:02.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0da7e640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf081029a0 0x7faf0819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:02.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0d27d640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf08100e90 0x7faf08069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:02.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0ca7c640 1 -- 172.21.15.72:0/1588549094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf08100e90 msgr2=0x7faf08069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0ca7c640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf08100e90 0x7faf08069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0ca7c640 1 -- 172.21.15.72:0/1588549094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf081029a0 msgr2=0x7faf0819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0ca7c640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf081029a0 0x7faf0819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0ca7c640 1 -- 172.21.15.72:0/1588549094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0819fed0 con 0x7faf08101fe0 2024-04-25T13:11:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.156+0000 7faf0ca7c640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 0x7faf0819cfa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.157+0000 7faefe7fc640 1 -- 172.21.15.72:0/1588549094 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef803f030 con 0x7faf08101fe0 2024-04-25T13:11:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.157+0000 7faefe7fc640 1 -- 172.21.15.72:0/1588549094 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef8003cb0 con 0x7faf08101fe0 2024-04-25T13:11:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.157+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf081a0100 con 0x7faf08101fe0 2024-04-25T13:11:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.157+0000 7faefe7fc640 1 -- 172.21.15.72:0/1588549094 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8035070 con 0x7faf08101fe0 2024-04-25T13:11:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.157+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf081a0640 con 0x7faf08101fe0 2024-04-25T13:11:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.158+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf08108310 con 0x7faf08101fe0 2024-04-25T13:11:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.159+0000 7faefe7fc640 1 -- 172.21.15.72:0/1588549094 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faef8042030 con 0x7faf08101fe0 2024-04-25T13:11:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.159+0000 7faefe7fc640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7faef40a3c30 0x7faef40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.160+0000 7faf0d27d640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7faef40a3c30 0x7faef40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.160+0000 7faefe7fc640 1 -- 172.21.15.72:0/1588549094 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7faef80f0560 con 0x7faf08101fe0 2024-04-25T13:11:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.160+0000 7faf0d27d640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7faef40a3c30 0x7faef40a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faf04005fd0 tx=0x7faf04005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:02.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.164+0000 7faefe7fc640 1 -- 172.21.15.72:0/1588549094 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faef80bd3c0 con 0x7faf08101fe0 2024-04-25T13:11:02.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:02.347+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faf0802d770 con 0x7faf08101fe0 2024-04-25T13:11:02.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:02 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2378923579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:11:02.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:02 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/4022940477' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:11:02.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:02 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:02.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:02 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1588549094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:11:03.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:02 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2378923579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:11:03.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:02 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/4022940477' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:11:03.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:02 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:03.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:02 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1588549094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:11:03.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:02 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2378923579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:11:03.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:02 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/4022940477' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:11:03.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:02 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:03.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:02 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1588549094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:11:03.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.252+0000 7faefe7fc640 1 -- 172.21.15.72:0/1588549094 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7faef80bccc0 con 0x7faf08101fe0 2024-04-25T13:11:03.252 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' created 2024-04-25T13:11:03.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7faef40a3c30 msgr2=0x7faef40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:03.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7faef40a3c30 0x7faef40a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faf04005fd0 tx=0x7faf04005850 comp rx=0 tx=0).stop 2024-04-25T13:11:03.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 msgr2=0x7faf0819cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:03.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 0x7faf0819cfa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef80042f0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 shutdown_connections 2024-04-25T13:11:03.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7faef40a3c30 0x7faef40a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf081029a0 0x7faf0819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf08101fe0 0x7faf0819cfa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 --2- 172.21.15.72:0/1588549094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf08100e90 0x7faf08069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 >> 172.21.15.72:0/1588549094 conn(0x7faf080fc400 msgr2=0x7faf081054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:03.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 shutdown_connections 2024-04-25T13:11:03.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.254+0000 7faf0f4e1640 1 -- 172.21.15.72:0/1588549094 wait complete. 2024-04-25T13:11:03.268 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.374+0000 7f2695059640 1 -- 172.21.15.72:0/2111900255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 msgr2=0x7f26901023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.374+0000 7f2695059640 1 --2- 172.21.15.72:0/2111900255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 0x7f26901023c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2680009900 tx=0x7f268002a410 comp rx=0 tx=0).stop 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.374+0000 7f2695059640 1 -- 172.21.15.72:0/2111900255 shutdown_connections 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.374+0000 7f2695059640 1 --2- 172.21.15.72:0/2111900255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2690103b50 0x7f269010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.374+0000 7f2695059640 1 --2- 172.21.15.72:0/2111900255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690103190 0x7f2690103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.374+0000 7f2695059640 1 --2- 172.21.15.72:0/2111900255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 0x7f26901023c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.374+0000 7f2695059640 1 -- 172.21.15.72:0/2111900255 >> 172.21.15.72:0/2111900255 conn(0x7f26900fd530 msgr2=0x7f26900ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.374+0000 7f2695059640 1 -- 172.21.15.72:0/2111900255 shutdown_connections 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.374+0000 7f2695059640 1 -- 172.21.15.72:0/2111900255 wait complete. 2024-04-25T13:11:03.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f2695059640 1 Processor -- start 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f2695059640 1 -- start start 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f2695059640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 0x7f2690198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f2695059640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2690103190 0x7f2690198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f2695059640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690103b50 0x7f269019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f2695059640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f269010dae0 con 0x7f2690101fa0 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f2695059640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f269010d960 con 0x7f2690103b50 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f2695059640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f269010dc60 con 0x7f2690103190 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 0x7f2690198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 0x7f2690198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42448/0 (socket says 172.21.15.72:42448) 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268ed9d640 1 -- 172.21.15.72:0/2816187058 learned_addr learned my addr 172.21.15.72:0/2816187058 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f2687fff640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2690103190 0x7f2690198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268f59e640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690103b50 0x7f269019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:03.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268ed9d640 1 -- 172.21.15.72:0/2816187058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2690103190 msgr2=0x7f2690198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268ed9d640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2690103190 0x7f2690198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268ed9d640 1 -- 172.21.15.72:0/2816187058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690103b50 msgr2=0x7f269019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268ed9d640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690103b50 0x7f269019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268ed9d640 1 -- 172.21.15.72:0/2816187058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269019fe30 con 0x7f2690101fa0 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268f59e640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690103b50 0x7f269019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268ed9d640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 0x7f2690198210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2680006010 tx=0x7f26800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.375+0000 7f268cd99640 1 -- 172.21.15.72:0/2816187058 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268003f030 con 0x7f2690101fa0 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.376+0000 7f268cd99640 1 -- 172.21.15.72:0/2816187058 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2680003cb0 con 0x7f2690101fa0 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.376+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26901a00c0 con 0x7f2690101fa0 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.376+0000 7f268cd99640 1 -- 172.21.15.72:0/2816187058 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680035070 con 0x7f2690101fa0 2024-04-25T13:11:03.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.376+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26901a05b0 con 0x7f2690101fa0 2024-04-25T13:11:03.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.377+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2690109380 con 0x7f2690101fa0 2024-04-25T13:11:03.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.378+0000 7f268cd99640 1 -- 172.21.15.72:0/2816187058 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2680042030 con 0x7f2690101fa0 2024-04-25T13:11:03.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.379+0000 7f268cd99640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f267c0a3bb0 0x7f267c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:03.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.379+0000 7f2687fff640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f267c0a3bb0 0x7f267c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:03.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.379+0000 7f268cd99640 1 -- 172.21.15.72:0/2816187058 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f26800f0ba0 con 0x7f2690101fa0 2024-04-25T13:11:03.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.379+0000 7f2687fff640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f267c0a3bb0 0x7f267c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2690199720 tx=0x7f267800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:03.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.383+0000 7f268cd99640 1 -- 172.21.15.72:0/2816187058 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26800bd810 con 0x7f2690101fa0 2024-04-25T13:11:03.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:03.560+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f269002d5c0 con 0x7f2690101fa0 2024-04-25T13:11:04.108 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.108 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1588549094' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:11:04.109 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:03 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2816187058' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:04.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1588549094' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:11:04.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:11:04.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:03 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2816187058' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:11:04.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.255+0000 7f268cd99640 1 -- 172.21.15.72:0/2816187058 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7f26800bd110 con 0x7f2690101fa0 2024-04-25T13:11:04.254 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' created 2024-04-25T13:11:04.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f267c0a3bb0 msgr2=0x7f267c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:04.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f267c0a3bb0 0x7f267c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2690199720 tx=0x7f267800b390 comp rx=0 tx=0).stop 2024-04-25T13:11:04.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 msgr2=0x7f2690198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:04.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 0x7f2690198210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2680006010 tx=0x7f26800042f0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 shutdown_connections 2024-04-25T13:11:04.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f267c0a3bb0 0x7f267c0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690103b50 0x7f269019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2690103190 0x7f2690198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 --2- 172.21.15.72:0/2816187058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2690101fa0 0x7f2690198210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 >> 172.21.15.72:0/2816187058 conn(0x7f26900fd530 msgr2=0x7f2690106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:04.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 shutdown_connections 2024-04-25T13:11:04.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.257+0000 7f2695059640 1 -- 172.21.15.72:0/2816187058 wait complete. 2024-04-25T13:11:04.275 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:11:04.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1588549094' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:11:04.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:03 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2816187058' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:11:04.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.377+0000 7fd325c74640 1 -- 172.21.15.72:0/3689475007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320103b50 msgr2=0x7fd32010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.377+0000 7fd325c74640 1 --2- 172.21.15.72:0/3689475007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320103b50 0x7fd32010a3e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd318009ca0 tx=0x7fd31802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 -- 172.21.15.72:0/3689475007 shutdown_connections 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 --2- 172.21.15.72:0/3689475007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320103b50 0x7fd32010a3e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 --2- 172.21.15.72:0/3689475007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320103190 0x7fd320103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 --2- 172.21.15.72:0/3689475007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd320101fa0 0x7fd3201023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 -- 172.21.15.72:0/3689475007 >> 172.21.15.72:0/3689475007 conn(0x7fd3200fd530 msgr2=0x7fd3200ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 -- 172.21.15.72:0/3689475007 shutdown_connections 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 -- 172.21.15.72:0/3689475007 wait complete. 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 Processor -- start 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 -- start start 2024-04-25T13:11:04.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320101fa0 0x7fd3201981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd320103190 0x7fd320198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320103b50 0x7fd32019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010db20 con 0x7fd320101fa0 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010d9a0 con 0x7fd320103190 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.378+0000 7fd325c74640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010dca0 con 0x7fd320103b50 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320101fa0 0x7fd3201981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320101fa0 0x7fd3201981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42468/0 (socket says 172.21.15.72:42468) 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31f7fe640 1 -- 172.21.15.72:0/2325937249 learned_addr learned my addr 172.21.15.72:0/2325937249 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31effd640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd320103190 0x7fd320198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31ffff640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320103b50 0x7fd32019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31f7fe640 1 -- 172.21.15.72:0/2325937249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320103b50 msgr2=0x7fd32019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31f7fe640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320103b50 0x7fd32019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31f7fe640 1 -- 172.21.15.72:0/2325937249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd320103190 msgr2=0x7fd320198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:04.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31f7fe640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd320103190 0x7fd320198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31f7fe640 1 -- 172.21.15.72:0/2325937249 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32019fe10 con 0x7fd320101fa0 2024-04-25T13:11:04.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31ffff640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320103b50 0x7fd32019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:04.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31f7fe640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320101fa0 0x7fd3201981f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d770 tx=0x7fd31000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:04.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31cff9640 1 -- 172.21.15.72:0/2325937249 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310017070 con 0x7fd320101fa0 2024-04-25T13:11:04.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3201a0100 con 0x7fd320101fa0 2024-04-25T13:11:04.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31cff9640 1 -- 172.21.15.72:0/2325937249 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd310009cd0 con 0x7fd320101fa0 2024-04-25T13:11:04.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd31cff9640 1 -- 172.21.15.72:0/2325937249 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310003c60 con 0x7fd320101fa0 2024-04-25T13:11:04.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.379+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3201a0720 con 0x7fd320101fa0 2024-04-25T13:11:04.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.380+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd320109310 con 0x7fd320101fa0 2024-04-25T13:11:04.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.381+0000 7fd31cff9640 1 -- 172.21.15.72:0/2325937249 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd310004340 con 0x7fd320101fa0 2024-04-25T13:11:04.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.382+0000 7fd31cff9640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3080a3bb0 0x7fd3080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:04.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.382+0000 7fd31effd640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3080a3bb0 0x7fd3080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:04.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.382+0000 7fd31cff9640 1 -- 172.21.15.72:0/2325937249 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7fd3100cfff0 con 0x7fd320101fa0 2024-04-25T13:11:04.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.382+0000 7fd31effd640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3080a3bb0 0x7fd3080a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd320199700 tx=0x7fd314009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:04.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.386+0000 7fd31cff9640 1 -- 172.21.15.72:0/2325937249 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd31009cb80 con 0x7fd320101fa0 2024-04-25T13:11:04.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:04.600+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd32002d5c0 con 0x7fd320101fa0 2024-04-25T13:11:05.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.263+0000 7fd31cff9640 1 -- 172.21.15.72:0/2325937249 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd31009c480 con 0x7fd320101fa0 2024-04-25T13:11:05.263 INFO:teuthology.orchestra.run.smithi072.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T13:11:05.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3080a3bb0 msgr2=0x7fd3080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3080a3bb0 0x7fd3080a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd320199700 tx=0x7fd314009110 comp rx=0 tx=0).stop 2024-04-25T13:11:05.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320101fa0 msgr2=0x7fd3201981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320101fa0 0x7fd3201981f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d770 tx=0x7fd31000db70 comp rx=0 tx=0).stop 2024-04-25T13:11:05.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 shutdown_connections 2024-04-25T13:11:05.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3080a3bb0 0x7fd3080a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320103b50 0x7fd32019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd320103190 0x7fd320198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 --2- 172.21.15.72:0/2325937249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd320101fa0 0x7fd3201981f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 >> 172.21.15.72:0/2325937249 conn(0x7fd3200fd530 msgr2=0x7fd3201064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:05.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 shutdown_connections 2024-04-25T13:11:05.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.267+0000 7fd325c74640 1 -- 172.21.15.72:0/2325937249 wait complete. 2024-04-25T13:11:05.287 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:11:05.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.386+0000 7f4e87d4b640 1 -- 172.21.15.72:0/643368994 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80101fa0 msgr2=0x7f4e801023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.386+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/643368994 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80101fa0 0x7f4e801023c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009ca0 tx=0x7f4e7802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.386+0000 7f4e87d4b640 1 -- 172.21.15.72:0/643368994 shutdown_connections 2024-04-25T13:11:05.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.386+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/643368994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e80103b50 0x7f4e8010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.386+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/643368994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e80103190 0x7f4e80103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.386+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/643368994 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80101fa0 0x7f4e801023c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.386+0000 7f4e87d4b640 1 -- 172.21.15.72:0/643368994 >> 172.21.15.72:0/643368994 conn(0x7f4e800fd530 msgr2=0x7f4e800ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 -- 172.21.15.72:0/643368994 shutdown_connections 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 -- 172.21.15.72:0/643368994 wait complete. 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 Processor -- start 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 -- start start 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e80101fa0 0x7f4e80198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e80103190 0x7f4e80198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80103b50 0x7f4e8019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8010dae0 con 0x7f4e80103b50 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8010d960 con 0x7f4e80103190 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e87d4b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8010dc60 con 0x7f4e80101fa0 2024-04-25T13:11:05.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e862e8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80103b50 0x7f4e8019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e862e8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80103b50 0x7f4e8019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42482/0 (socket says 172.21.15.72:42482) 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e862e8640 1 -- 172.21.15.72:0/1578075003 learned_addr learned my addr 172.21.15.72:0/1578075003 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.387+0000 7f4e85ae7640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e80101fa0 0x7f4e80198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e852e6640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e80103190 0x7f4e80198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e862e8640 1 -- 172.21.15.72:0/1578075003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e80101fa0 msgr2=0x7f4e80198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e862e8640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e80101fa0 0x7f4e80198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e862e8640 1 -- 172.21.15.72:0/1578075003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e80103190 msgr2=0x7f4e80198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e862e8640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e80103190 0x7f4e80198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e862e8640 1 -- 172.21.15.72:0/1578075003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8019fe20 con 0x7f4e80103b50 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e862e8640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80103b50 0x7f4e8019f7b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c970 tx=0x7f4e7c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e76ffd640 1 -- 172.21.15.72:0/1578075003 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c007bb0 con 0x7f4e80103b50 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e76ffd640 1 -- 172.21.15.72:0/1578075003 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e7c007d50 con 0x7f4e80103b50 2024-04-25T13:11:05.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e801a0110 con 0x7f4e80103b50 2024-04-25T13:11:05.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e76ffd640 1 -- 172.21.15.72:0/1578075003 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c004e60 con 0x7f4e80103b50 2024-04-25T13:11:05.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.388+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801a0660 con 0x7f4e80103b50 2024-04-25T13:11:05.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.389+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e801093e0 con 0x7f4e80103b50 2024-04-25T13:11:05.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.390+0000 7f4e76ffd640 1 -- 172.21.15.72:0/1578075003 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4e7c005000 con 0x7f4e80103b50 2024-04-25T13:11:05.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.390+0000 7f4e76ffd640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4e6c0a3bb0 0x7f4e6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.391+0000 7f4e85ae7640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4e6c0a3bb0 0x7f4e6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.391+0000 7f4e76ffd640 1 -- 172.21.15.72:0/1578075003 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f4e7c0d0330 con 0x7f4e80103b50 2024-04-25T13:11:05.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.391+0000 7f4e85ae7640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4e6c0a3bb0 0x7f4e6c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4e78000f80 tx=0x7f4e780093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:05.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.394+0000 7f4e76ffd640 1 -- 172.21.15.72:0/1578075003 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e7c014030 con 0x7f4e80103b50 2024-04-25T13:11:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:05 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2816187058' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:11:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:05 smithi072 ceph-mon[29199]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:11:05.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:05 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2325937249' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:11:05.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:05 smithi072 ceph-mon[29199]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:11:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:05 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2816187058' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:11:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:05 smithi164 ceph-mon[32418]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:11:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:05 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2325937249' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:11:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:05 smithi164 ceph-mon[32418]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:11:05.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.573+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e80062b50 con 0x7f4e80103b50 2024-04-25T13:11:05.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.575+0000 7f4e76ffd640 1 -- 172.21.15.72:0/1578075003 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22256 (secure 0 0 0) 0x7f4e7c09ce60 con 0x7f4e80103b50 2024-04-25T13:11:05.575 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:05.575 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":87,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:11:04.601676+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:11:05.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4e6c0a3bb0 msgr2=0x7f4e6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4e6c0a3bb0 0x7f4e6c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4e78000f80 tx=0x7f4e780093f0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80103b50 msgr2=0x7f4e8019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80103b50 0x7f4e8019f7b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c970 tx=0x7f4e7c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:11:05.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 shutdown_connections 2024-04-25T13:11:05.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4e6c0a3bb0 0x7f4e6c0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e80103b50 0x7f4e8019f7b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e80103190 0x7f4e80198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 --2- 172.21.15.72:0/1578075003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e80101fa0 0x7f4e80198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 >> 172.21.15.72:0/1578075003 conn(0x7f4e800fd530 msgr2=0x7f4e801065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:05.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 shutdown_connections 2024-04-25T13:11:05.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.577+0000 7f4e87d4b640 1 -- 172.21.15.72:0/1578075003 wait complete. 2024-04-25T13:11:05.588 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:11:05.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 -- 172.21.15.72:0/3260149164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 msgr2=0x7f09f810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/3260149164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 0x7f09f810a0e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009ca0 tx=0x7f09f002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 -- 172.21.15.72:0/3260149164 shutdown_connections 2024-04-25T13:11:05.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/3260149164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 0x7f09f810a0e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/3260149164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09f81053c0 0x7f09f81077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/3260149164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f8102a90 0x7f09f8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 -- 172.21.15.72:0/3260149164 >> 172.21.15.72:0/3260149164 conn(0x7f09f80fc440 msgr2=0x7f09f80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 -- 172.21.15.72:0/3260149164 shutdown_connections 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 -- 172.21.15.72:0/3260149164 wait complete. 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.686+0000 7f09fc9b4640 1 Processor -- start 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09fc9b4640 1 -- start start 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09fc9b4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09f8102a90 0x7f09f8198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09fc9b4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f81053c0 0x7f09f8198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09fc9b4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 0x7f09f819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09fc9b4640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810daf0 con 0x7f09f8107cf0 2024-04-25T13:11:05.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09fc9b4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810d970 con 0x7f09f81053c0 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09fc9b4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810dc70 con 0x7f09f8102a90 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f6d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 0x7f09f819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f6d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 0x7f09f819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56188/0 (socket says 172.21.15.72:56188) 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f6d9d640 1 -- 172.21.15.72:0/2435264478 learned_addr learned my addr 172.21.15.72:0/2435264478 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f659c640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09f8102a90 0x7f09f8198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f5d9b640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f81053c0 0x7f09f8198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f6d9d640 1 -- 172.21.15.72:0/2435264478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09f8102a90 msgr2=0x7f09f8198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f6d9d640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09f8102a90 0x7f09f8198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f6d9d640 1 -- 172.21.15.72:0/2435264478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f81053c0 msgr2=0x7f09f8198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f6d9d640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f81053c0 0x7f09f8198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f6d9d640 1 -- 172.21.15.72:0/2435264478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f819fd60 con 0x7f09f8107cf0 2024-04-25T13:11:05.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f5d9b640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f81053c0 0x7f09f8198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:05.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f659c640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09f8102a90 0x7f09f8198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:11:05.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09f6d9d640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 0x7f09f819f6f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f09f0031a20 tx=0x7f09f0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:05.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09e77fe640 1 -- 172.21.15.72:0/2435264478 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0004240 con 0x7f09f8107cf0 2024-04-25T13:11:05.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09e77fe640 1 -- 172.21.15.72:0/2435264478 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09f0003e10 con 0x7f09f8107cf0 2024-04-25T13:11:05.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f819fff0 con 0x7f09f8107cf0 2024-04-25T13:11:05.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09e77fe640 1 -- 172.21.15.72:0/2435264478 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f00305c0 con 0x7f09f8107cf0 2024-04-25T13:11:05.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.687+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f81a0500 con 0x7f09f8107cf0 2024-04-25T13:11:05.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.688+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f80fdf30 con 0x7f09f8107cf0 2024-04-25T13:11:05.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.690+0000 7f09e77fe640 1 -- 172.21.15.72:0/2435264478 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f09f002f660 con 0x7f09f8107cf0 2024-04-25T13:11:05.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.690+0000 7f09e77fe640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f09e00a3c30 0x7f09e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.690+0000 7f09f659c640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f09e00a3c30 0x7f09e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.690+0000 7f09e77fe640 1 -- 172.21.15.72:0/2435264478 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f09f00f1d10 con 0x7f09f8107cf0 2024-04-25T13:11:05.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.691+0000 7f09f659c640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f09e00a3c30 0x7f09e00a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00abe0 tx=0x7f09ec00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:05.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.694+0000 7f09e77fe640 1 -- 172.21.15.72:0/2435264478 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09f00be840 con 0x7f09f8107cf0 2024-04-25T13:11:05.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:05 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2816187058' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:11:05.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:05 smithi193 ceph-mon[32743]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:11:05.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:05 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2325937249' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:11:05.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:05 smithi193 ceph-mon[32743]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:11:05.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.872+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09f802d810 con 0x7f09f8107cf0 2024-04-25T13:11:05.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.874+0000 7f09e77fe640 1 -- 172.21.15.72:0/2435264478 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22256 (secure 0 0 0) 0x7f09f00be140 con 0x7f09f8107cf0 2024-04-25T13:11:05.874 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:05.874 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":87,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:11:04.601676+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:11:05.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f09e00a3c30 msgr2=0x7f09e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f09e00a3c30 0x7f09e00a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00abe0 tx=0x7f09ec00a5c0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 msgr2=0x7f09f819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 0x7f09f819f6f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f09f0031a20 tx=0x7f09f0003c20 comp rx=0 tx=0).stop 2024-04-25T13:11:05.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 shutdown_connections 2024-04-25T13:11:05.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f09e00a3c30 0x7f09e00a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09f8107cf0 0x7f09f819f6f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f81053c0 0x7f09f8198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 --2- 172.21.15.72:0/2435264478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09f8102a90 0x7f09f8198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 >> 172.21.15.72:0/2435264478 conn(0x7f09f80fc440 msgr2=0x7f09f81033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:05.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 shutdown_connections 2024-04-25T13:11:05.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.876+0000 7f09fc9b4640 1 -- 172.21.15.72:0/2435264478 wait complete. 2024-04-25T13:11:05.887 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:11:05.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.988+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/3792733059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 msgr2=0x7f8e28107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.988+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/3792733059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 0x7f8e28107750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009900 tx=0x7f8e1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:11:05.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.988+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/3792733059 shutdown_connections 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.988+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/3792733059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e28107c90 0x7f8e2810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.988+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/3792733059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 0x7f8e28107750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.988+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/3792733059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e28102a30 0x7f8e28104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.988+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/3792733059 >> 172.21.15.72:0/3792733059 conn(0x7f8e280fc440 msgr2=0x7f8e280fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.988+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/3792733059 shutdown_connections 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.988+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/3792733059 wait complete. 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2f1cd640 1 Processor -- start 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2f1cd640 1 -- start start 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2f1cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e28102a30 0x7f8e28198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2f1cd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 0x7f8e28198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2f1cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e28107c90 0x7f8e2819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2f1cd640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2810db30 con 0x7f8e28105360 2024-04-25T13:11:05.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2f1cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2810d9b0 con 0x7f8e28107c90 2024-04-25T13:11:05.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2f1cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2810dcb0 con 0x7f8e28102a30 2024-04-25T13:11:05.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 0x7f8e28198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 0x7f8e28198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56202/0 (socket says 172.21.15.72:56202) 2024-04-25T13:11:05.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e27fff640 1 -- 172.21.15.72:0/50498340 learned_addr learned my addr 172.21.15.72:0/50498340 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:05.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2d76a640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e28107c90 0x7f8e2819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e27fff640 1 -- 172.21.15.72:0/50498340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e28102a30 msgr2=0x7f8e28198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e27fff640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e28102a30 0x7f8e28198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e27fff640 1 -- 172.21.15.72:0/50498340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e28107c90 msgr2=0x7f8e2819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:05.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e27fff640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e28107c90 0x7f8e2819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:05.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e27fff640 1 -- 172.21.15.72:0/50498340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2819fe20 con 0x7f8e28105360 2024-04-25T13:11:05.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e2d76a640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e28107c90 0x7f8e2819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:05.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.989+0000 7f8e27fff640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 0x7f8e28198740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c006010 tx=0x7f8e1c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:05.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.990+0000 7f8e25ffb640 1 -- 172.21.15.72:0/50498340 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c03f030 con 0x7f8e28105360 2024-04-25T13:11:05.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.990+0000 7f8e25ffb640 1 -- 172.21.15.72:0/50498340 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1c003cb0 con 0x7f8e28105360 2024-04-25T13:11:05.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.990+0000 7f8e25ffb640 1 -- 172.21.15.72:0/50498340 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c035070 con 0x7f8e28105360 2024-04-25T13:11:05.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.990+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e281a00b0 con 0x7f8e28105360 2024-04-25T13:11:05.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.990+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e281a0670 con 0x7f8e28105360 2024-04-25T13:11:05.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.991+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e280fdf30 con 0x7f8e28105360 2024-04-25T13:11:05.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.992+0000 7f8e25ffb640 1 -- 172.21.15.72:0/50498340 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e1c042030 con 0x7f8e28105360 2024-04-25T13:11:05.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.992+0000 7f8e25ffb640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8e100a3c30 0x7f8e100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:05.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.993+0000 7f8e2cf69640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8e100a3c30 0x7f8e100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:05.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.993+0000 7f8e25ffb640 1 -- 172.21.15.72:0/50498340 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f8e1c0f09f0 con 0x7f8e28105360 2024-04-25T13:11:05.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.993+0000 7f8e2cf69640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8e100a3c30 0x7f8e100a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800a990 tx=0x7f8e18005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:05.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:05.997+0000 7f8e25ffb640 1 -- 172.21.15.72:0/50498340 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e1c003e60 con 0x7f8e28105360 2024-04-25T13:11:06.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.208+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8e2802d850 con 0x7f8e28105360 2024-04-25T13:11:06.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.269+0000 7f8e25ffb640 1 -- 172.21.15.72:0/50498340 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f8e1c0bd520 con 0x7f8e28105360 2024-04-25T13:11:06.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.271+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8e100a3c30 msgr2=0x7f8e100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:06.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.271+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8e100a3c30 0x7f8e100a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800a990 tx=0x7f8e18005c70 comp rx=0 tx=0).stop 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.271+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 msgr2=0x7f8e28198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.271+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 0x7f8e28198740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c006010 tx=0x7f8e1c004430 comp rx=0 tx=0).stop 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.271+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 shutdown_connections 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.271+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f8e100a3c30 0x7f8e100a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.271+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e28107c90 0x7f8e2819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.271+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e28105360 0x7f8e28198740 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.271+0000 7f8e2f1cd640 1 --2- 172.21.15.72:0/50498340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e28102a30 0x7f8e28198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.272+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 >> 172.21.15.72:0/50498340 conn(0x7f8e280fc440 msgr2=0x7f8e28103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.272+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 shutdown_connections 2024-04-25T13:11:06.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.272+0000 7f8e2f1cd640 1 -- 172.21.15.72:0/50498340 wait complete. 2024-04-25T13:11:06.283 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T13:11:06.283 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T13:11:06.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.383+0000 7f04a35ed640 1 -- 172.21.15.72:0/1869355491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 msgr2=0x7f049c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:06.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.383+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1869355491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 0x7f049c1092b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0494009ca0 tx=0x7f049402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.383+0000 7f04a35ed640 1 -- 172.21.15.72:0/1869355491 shutdown_connections 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.383+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1869355491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 0x7f049c1092b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.383+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1869355491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049c101fd0 0x7f049c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.383+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1869355491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c100e80 0x7f049c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.383+0000 7f04a35ed640 1 -- 172.21.15.72:0/1869355491 >> 172.21.15.72:0/1869355491 conn(0x7f049c0fc3f0 msgr2=0x7f049c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.383+0000 7f04a35ed640 1 -- 172.21.15.72:0/1869355491 shutdown_connections 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.383+0000 7f04a35ed640 1 -- 172.21.15.72:0/1869355491 wait complete. 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a35ed640 1 Processor -- start 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a35ed640 1 -- start start 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a35ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049c100e80 0x7f049c06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a35ed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c101fd0 0x7f049c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a35ed640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 0x7f049c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a35ed640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f049c10ca50 con 0x7f049c102990 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a35ed640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f049c10c8d0 con 0x7f049c100e80 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a35ed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f049c10cbd0 con 0x7f049c101fd0 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1b8a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 0x7f049c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1b8a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 0x7f049c19f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56214/0 (socket says 172.21.15.72:56214) 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1b8a640 1 -- 172.21.15.72:0/1208816172 learned_addr learned my addr 172.21.15.72:0/1208816172 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1389640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049c100e80 0x7f049c06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a0b88640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c101fd0 0x7f049c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1b8a640 1 -- 172.21.15.72:0/1208816172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c101fd0 msgr2=0x7f049c19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1b8a640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c101fd0 0x7f049c19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1b8a640 1 -- 172.21.15.72:0/1208816172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049c100e80 msgr2=0x7f049c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1b8a640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049c100e80 0x7f049c06b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1b8a640 1 -- 172.21.15.72:0/1208816172 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c19fe60 con 0x7f049c102990 2024-04-25T13:11:06.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a1b8a640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 0x7f049c19f550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0494031a20 tx=0x7f0494003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:06.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04927fc640 1 -- 172.21.15.72:0/1208816172 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494004240 con 0x7f049c102990 2024-04-25T13:11:06.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04927fc640 1 -- 172.21.15.72:0/1208816172 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0494003e10 con 0x7f049c102990 2024-04-25T13:11:06.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049c1a0090 con 0x7f049c102990 2024-04-25T13:11:06.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.384+0000 7f04927fc640 1 -- 172.21.15.72:0/1208816172 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940305c0 con 0x7f049c102990 2024-04-25T13:11:06.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.385+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f049c1a0660 con 0x7f049c102990 2024-04-25T13:11:06.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.385+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f049c108260 con 0x7f049c102990 2024-04-25T13:11:06.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.386+0000 7f04927fc640 1 -- 172.21.15.72:0/1208816172 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f049402f660 con 0x7f049c102990 2024-04-25T13:11:06.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.387+0000 7f04927fc640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04880a3bb0 0x7f04880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:06.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.387+0000 7f04927fc640 1 -- 172.21.15.72:0/1208816172 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f04940f1c40 con 0x7f049c102990 2024-04-25T13:11:06.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.387+0000 7f04a1389640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04880a3bb0 0x7f04880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:06.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.387+0000 7f04a1389640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04880a3bb0 0x7f04880a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04980058c0 tx=0x7f049800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.392+0000 7f04927fc640 1 -- 172.21.15.72:0/1208816172 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04940be770 con 0x7f049c102990 2024-04-25T13:11:06.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:06 smithi072 ceph-mon[29199]: pgmap v285: 97 pgs: 5 active+clean, 17 creating+peering, 75 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:06.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:06 smithi072 ceph-mon[29199]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:11:06.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:06 smithi072 ceph-mon[29199]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:11:06.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:06 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2325937249' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:11:06.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:06 smithi072 ceph-mon[29199]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:11:06.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:06 smithi072 ceph-mon[29199]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:11:06.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:06 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1578075003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:06.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:06 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2435264478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:06.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:06 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/50498340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:11:06.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:06 smithi164 ceph-mon[32418]: pgmap v285: 97 pgs: 5 active+clean, 17 creating+peering, 75 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:06.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:06 smithi164 ceph-mon[32418]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:11:06.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:06 smithi164 ceph-mon[32418]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:11:06.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:06 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2325937249' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:11:06.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:06 smithi164 ceph-mon[32418]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:11:06.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:06 smithi164 ceph-mon[32418]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:11:06.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:06 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1578075003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:06.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:06 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2435264478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:06.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:06 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/50498340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:11:06.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:06.609+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f049c02d810 con 0x7f049c102990 2024-04-25T13:11:06.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:06 smithi193 ceph-mon[32743]: pgmap v285: 97 pgs: 5 active+clean, 17 creating+peering, 75 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:11:06.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:06 smithi193 ceph-mon[32743]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:11:06.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:06 smithi193 ceph-mon[32743]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:11:06.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:06 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2325937249' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:11:06.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:06 smithi193 ceph-mon[32743]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:11:06.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:06 smithi193 ceph-mon[32743]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:11:06.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:06 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1578075003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:06.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:06 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2435264478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:06.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:06 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/50498340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:11:07.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.253+0000 7f04927fc640 1 -- 172.21.15.72:0/1208816172 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f04940be070 con 0x7f049c102990 2024-04-25T13:11:07.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04880a3bb0 msgr2=0x7f04880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:07.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04880a3bb0 0x7f04880a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04980058c0 tx=0x7f049800a4c0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 msgr2=0x7f049c19f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:07.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 0x7f049c19f550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0494031a20 tx=0x7f0494003c20 comp rx=0 tx=0).stop 2024-04-25T13:11:07.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 shutdown_connections 2024-04-25T13:11:07.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04880a3bb0 0x7f04880a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f049c102990 0x7f049c19f550 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c101fd0 0x7f049c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 --2- 172.21.15.72:0/1208816172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049c100e80 0x7f049c06b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 >> 172.21.15.72:0/1208816172 conn(0x7f049c0fc3f0 msgr2=0x7f049c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 shutdown_connections 2024-04-25T13:11:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.255+0000 7f04a35ed640 1 -- 172.21.15.72:0/1208816172 wait complete. 2024-04-25T13:11:07.269 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T13:11:07.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.371+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/2786265426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4103b40 msgr2=0x7fe1b410a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:07.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.371+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/2786265426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4103b40 0x7fe1b410a3d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009ca0 tx=0x7fe1ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.371+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/2786265426 shutdown_connections 2024-04-25T13:11:07.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.371+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/2786265426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4103b40 0x7fe1b410a3d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.371+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/2786265426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1b4103180 0x7fe1b4103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.371+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/2786265426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101f90 0x7fe1b41023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.371+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/2786265426 >> 172.21.15.72:0/2786265426 conn(0x7fe1b40fd500 msgr2=0x7fe1b40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:07.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.371+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/2786265426 shutdown_connections 2024-04-25T13:11:07.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.371+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/2786265426 wait complete. 2024-04-25T13:11:07.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 Processor -- start 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 -- start start 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4101f90 0x7fe1b4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4103180 0x7fe1b4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1b4103b40 0x7fe1b419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410dbb0 con 0x7fe1b4101f90 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410da30 con 0x7fe1b4103180 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410dd30 con 0x7fe1b4103b40 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4101f90 0x7fe1b4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4101f90 0x7fe1b4198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56244/0 (socket says 172.21.15.72:56244) 2024-04-25T13:11:07.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b259c640 1 -- 172.21.15.72:0/1956766110 learned_addr learned my addr 172.21.15.72:0/1956766110 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b2d9d640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1b4103b40 0x7fe1b419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b259c640 1 -- 172.21.15.72:0/1956766110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1b4103b40 msgr2=0x7fe1b419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b259c640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1b4103b40 0x7fe1b419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b259c640 1 -- 172.21.15.72:0/1956766110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4103180 msgr2=0x7fe1b4198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b259c640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4103180 0x7fe1b4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b259c640 1 -- 172.21.15.72:0/1956766110 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b419ff50 con 0x7fe1b4101f90 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b259c640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4101f90 0x7fe1b4198330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00b450 tx=0x7fe19c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1ab7fe640 1 -- 172.21.15.72:0/1956766110 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c015020 con 0x7fe1b4101f90 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1ab7fe640 1 -- 172.21.15.72:0/1956766110 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe19c011d40 con 0x7fe1b4101f90 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b41a0240 con 0x7fe1b4101f90 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1ab7fe640 1 -- 172.21.15.72:0/1956766110 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c00f340 con 0x7fe1b4101f90 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.372+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b41a0810 con 0x7fe1b4101f90 2024-04-25T13:11:07.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.374+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b41093d0 con 0x7fe1b4101f90 2024-04-25T13:11:07.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.375+0000 7fe1ab7fe640 1 -- 172.21.15.72:0/1956766110 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe19c018020 con 0x7fe1b4101f90 2024-04-25T13:11:07.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.375+0000 7fe1ab7fe640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1a00a3bb0 0x7fe1a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:07.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.375+0000 7fe1b1d9b640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1a00a3bb0 0x7fe1a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:07.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.375+0000 7fe1ab7fe640 1 -- 172.21.15.72:0/1956766110 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fe19c0cfdf0 con 0x7fe1b4101f90 2024-04-25T13:11:07.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.376+0000 7fe1b1d9b640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1a00a3bb0 0x7fe1a00a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4199840 tx=0x7fe1a4006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:07.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.380+0000 7fe1ab7fe640 1 -- 172.21.15.72:0/1956766110 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe19c01f020 con 0x7fe1b4101f90 2024-04-25T13:11:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:07 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/50498340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:11:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:07 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] up:active 2024-04-25T13:11:07.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:07 smithi072 ceph-mon[29199]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:11:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:07 smithi072 ceph-mon[29199]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:11:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:07 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1208816172' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T13:11:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:07 smithi072 ceph-mon[29199]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:11:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:07 smithi072 ceph-mon[29199]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:11:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:07 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1208816172' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T13:11:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:07 smithi072 ceph-mon[29199]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:11:07.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:07 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/50498340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:11:07.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:07 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] up:active 2024-04-25T13:11:07.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:07 smithi164 ceph-mon[32418]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:11:07.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:07 smithi164 ceph-mon[32418]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:11:07.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:07 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1208816172' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T13:11:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:07 smithi164 ceph-mon[32418]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:11:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:07 smithi164 ceph-mon[32418]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:11:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:07 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1208816172' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T13:11:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:07 smithi164 ceph-mon[32418]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:11:07.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:07.595+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe1b402d810 con 0x7fe1b4101f90 2024-04-25T13:11:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:07 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/50498340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:11:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:07 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] up:active 2024-04-25T13:11:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:07 smithi193 ceph-mon[32743]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:11:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:07 smithi193 ceph-mon[32743]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:11:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:07 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1208816172' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T13:11:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:07 smithi193 ceph-mon[32743]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:11:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:07 smithi193 ceph-mon[32743]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:11:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:07 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1208816172' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T13:11:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:07 smithi193 ceph-mon[32743]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:11:08.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.274+0000 7fe1ab7fe640 1 -- 172.21.15.72:0/1956766110 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7fe19c00d040 con 0x7fe1b4101f90 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1a00a3bb0 msgr2=0x7fe1a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1a00a3bb0 0x7fe1a00a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4199840 tx=0x7fe1a4006c90 comp rx=0 tx=0).stop 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4101f90 msgr2=0x7fe1b4198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4101f90 0x7fe1b4198330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00b450 tx=0x7fe19c00b910 comp rx=0 tx=0).stop 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 shutdown_connections 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1a00a3bb0 0x7fe1a00a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1b4103b40 0x7fe1b419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4103180 0x7fe1b4198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 --2- 172.21.15.72:0/1956766110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1b4101f90 0x7fe1b4198330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 >> 172.21.15.72:0/1956766110 conn(0x7fe1b40fd500 msgr2=0x7fe1b41065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 shutdown_connections 2024-04-25T13:11:08.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.278+0000 7fe1b8ae8640 1 -- 172.21.15.72:0/1956766110 wait complete. 2024-04-25T13:11:08.296 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T13:11:08.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.401+0000 7f986614f640 1 -- 172.21.15.72:0/1829833583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860101fa0 msgr2=0x7f98601023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:08.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.401+0000 7f986614f640 1 --2- 172.21.15.72:0/1829833583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860101fa0 0x7f98601023c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9858009ca0 tx=0x7f985802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.401+0000 7f986614f640 1 -- 172.21.15.72:0/1829833583 shutdown_connections 2024-04-25T13:11:08.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.401+0000 7f986614f640 1 --2- 172.21.15.72:0/1829833583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860103b50 0x7f986010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.401+0000 7f986614f640 1 --2- 172.21.15.72:0/1829833583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9860103190 0x7f9860103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.401+0000 7f986614f640 1 --2- 172.21.15.72:0/1829833583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860101fa0 0x7f98601023c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.401+0000 7f986614f640 1 -- 172.21.15.72:0/1829833583 >> 172.21.15.72:0/1829833583 conn(0x7f98600fd530 msgr2=0x7f98600ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:08.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.401+0000 7f986614f640 1 -- 172.21.15.72:0/1829833583 shutdown_connections 2024-04-25T13:11:08.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.401+0000 7f986614f640 1 -- 172.21.15.72:0/1829833583 wait complete. 2024-04-25T13:11:08.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f986614f640 1 Processor -- start 2024-04-25T13:11:08.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f986614f640 1 -- start start 2024-04-25T13:11:08.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f986614f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860101fa0 0x7f9860198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:08.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f986614f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9860103190 0x7f98601987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:08.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f986614f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860103b50 0x7f986019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:08.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f986614f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f986010daa0 con 0x7f9860103b50 2024-04-25T13:11:08.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f986614f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f986010d920 con 0x7f9860103190 2024-04-25T13:11:08.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f986614f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f986010dc20 con 0x7f9860101fa0 2024-04-25T13:11:08.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f985ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860103b50 0x7f986019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:08.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f985ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860103b50 0x7f986019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56270/0 (socket says 172.21.15.72:56270) 2024-04-25T13:11:08.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f985ffff640 1 -- 172.21.15.72:0/2738224329 learned_addr learned my addr 172.21.15.72:0/2738224329 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:08.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f985effd640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9860103190 0x7f98601987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:08.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f985f7fe640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860101fa0 0x7f9860198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:08.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f985ffff640 1 -- 172.21.15.72:0/2738224329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860101fa0 msgr2=0x7f9860198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.402+0000 7f985ffff640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860101fa0 0x7f9860198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.403+0000 7f985ffff640 1 -- 172.21.15.72:0/2738224329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9860103190 msgr2=0x7f98601987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.403+0000 7f985ffff640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9860103190 0x7f98601987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.403+0000 7f985ffff640 1 -- 172.21.15.72:0/2738224329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986019fe80 con 0x7f9860103b50 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.403+0000 7f985ffff640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860103b50 0x7f986019f810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f985000c970 tx=0x7f985000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.403+0000 7f985cff9640 1 -- 172.21.15.72:0/2738224329 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850007bb0 con 0x7f9860103b50 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.403+0000 7f985cff9640 1 -- 172.21.15.72:0/2738224329 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9850007d50 con 0x7f9860103b50 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.403+0000 7f985cff9640 1 -- 172.21.15.72:0/2738224329 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850004e60 con 0x7f9860103b50 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.403+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98601a0170 con 0x7f9860103b50 2024-04-25T13:11:08.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.403+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98601a0670 con 0x7f9860103b50 2024-04-25T13:11:08.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.404+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9860109430 con 0x7f9860103b50 2024-04-25T13:11:08.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.405+0000 7f985cff9640 1 -- 172.21.15.72:0/2738224329 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9850005000 con 0x7f9860103b50 2024-04-25T13:11:08.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.406+0000 7f985cff9640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f984c0a3bb0 0x7f984c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:08.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.406+0000 7f985f7fe640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f984c0a3bb0 0x7f984c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:08.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.406+0000 7f985cff9640 1 -- 172.21.15.72:0/2738224329 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f98500d0290 con 0x7f9860103b50 2024-04-25T13:11:08.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.406+0000 7f985f7fe640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f984c0a3bb0 0x7f984c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9858000f80 tx=0x7f9858005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:08.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.409+0000 7f985cff9640 1 -- 172.21.15.72:0/2738224329 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9850014030 con 0x7f9860103b50 2024-04-25T13:11:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: pgmap v288: 97 pgs: 62 active+clean, 21 creating+peering, 14 unknown; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-25T13:11:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:11:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:11:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1956766110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:11:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1956766110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:11:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] up:active 2024-04-25T13:11:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:11:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:11:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: Cluster is now healthy 2024-04-25T13:11:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:08 smithi164 ceph-mon[32418]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T13:11:08.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:08.624+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f986002d5c0 con 0x7f9860103b50 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: pgmap v288: 97 pgs: 62 active+clean, 21 creating+peering, 14 unknown; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1956766110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1956766110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] up:active 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: Cluster is now healthy 2024-04-25T13:11:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:08 smithi193 ceph-mon[32743]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T13:11:08.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: pgmap v288: 97 pgs: 62 active+clean, 21 creating+peering, 14 unknown; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-25T13:11:08.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:11:08.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:11:08.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1956766110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:11:08.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1956766110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:11:08.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] up:active 2024-04-25T13:11:08.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:11:08.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:11:08.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: Cluster is now healthy 2024-04-25T13:11:08.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:08 smithi072 ceph-mon[29199]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T13:11:09.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.287+0000 7f985cff9640 1 -- 172.21.15.72:0/2738224329 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7f985009c770 con 0x7f9860103b50 2024-04-25T13:11:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f984c0a3bb0 msgr2=0x7f984c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f984c0a3bb0 0x7f984c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9858000f80 tx=0x7f9858005dc0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860103b50 msgr2=0x7f986019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860103b50 0x7f986019f810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f985000c970 tx=0x7f985000ce30 comp rx=0 tx=0).stop 2024-04-25T13:11:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 shutdown_connections 2024-04-25T13:11:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f984c0a3bb0 0x7f984c0a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9860103b50 0x7f986019f810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9860103190 0x7f98601987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 --2- 172.21.15.72:0/2738224329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860101fa0 0x7f9860198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 >> 172.21.15.72:0/2738224329 conn(0x7f98600fd530 msgr2=0x7f9860106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:09.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 shutdown_connections 2024-04-25T13:11:09.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.290+0000 7f986614f640 1 -- 172.21.15.72:0/2738224329 wait complete. 2024-04-25T13:11:09.302 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T13:11:09.303 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T13:11:09.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 -- 172.21.15.72:0/2976220637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 msgr2=0x7fadd0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 --2- 172.21.15.72:0/2976220637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 0x7fadd0109320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009ca0 tx=0x7fadc802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 -- 172.21.15.72:0/2976220637 shutdown_connections 2024-04-25T13:11:09.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 --2- 172.21.15.72:0/2976220637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 0x7fadd0109320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 --2- 172.21.15.72:0/2976220637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0102040 0x7fadd01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 --2- 172.21.15.72:0/2976220637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0100e60 0x7fadd0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 -- 172.21.15.72:0/2976220637 >> 172.21.15.72:0/2976220637 conn(0x7fadd00fc440 msgr2=0x7fadd00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 -- 172.21.15.72:0/2976220637 shutdown_connections 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 -- 172.21.15.72:0/2976220637 wait complete. 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 Processor -- start 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 -- start start 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0100e60 0x7fadd01970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0102040 0x7fadd01975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.406+0000 7fadd78e8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 0x7fadd019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd78e8640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fadd010ca10 con 0x7fadd0102a00 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd78e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fadd010c890 con 0x7fadd0102040 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd78e8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fadd010cb90 con 0x7fadd0100e60 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5e85640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 0x7fadd019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5e85640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 0x7fadd019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56294/0 (socket says 172.21.15.72:56294) 2024-04-25T13:11:09.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd4e83640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0102040 0x7fadd01975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5e85640 1 -- 172.21.15.72:0/21071546 learned_addr learned my addr 172.21.15.72:0/21071546 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5684640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0100e60 0x7fadd01970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5e85640 1 -- 172.21.15.72:0/21071546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0100e60 msgr2=0x7fadd01970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5e85640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0100e60 0x7fadd01970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5e85640 1 -- 172.21.15.72:0/21071546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0102040 msgr2=0x7fadd01975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5e85640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0102040 0x7fadd01975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5e85640 1 -- 172.21.15.72:0/21071546 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd019ecd0 con 0x7fadd0102a00 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5684640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0100e60 0x7fadd01970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd4e83640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0102040 0x7fadd01975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd5e85640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 0x7fadd019e660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fadc80097b0 tx=0x7fadc8031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadc67fc640 1 -- 172.21.15.72:0/21071546 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8003bf0 con 0x7fadd0102a00 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadc67fc640 1 -- 172.21.15.72:0/21071546 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc8003d90 con 0x7fadd0102a00 2024-04-25T13:11:09.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd006a590 con 0x7fadd0102a00 2024-04-25T13:11:09.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadc67fc640 1 -- 172.21.15.72:0/21071546 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc802fc70 con 0x7fadd0102a00 2024-04-25T13:11:09.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.407+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd006ab30 con 0x7fadd0102a00 2024-04-25T13:11:09.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.408+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd00691c0 con 0x7fadd0102a00 2024-04-25T13:11:09.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.409+0000 7fadc67fc640 1 -- 172.21.15.72:0/21071546 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fadc8004040 con 0x7fadd0102a00 2024-04-25T13:11:09.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.410+0000 7fadc67fc640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fadbc0a3c30 0x7fadbc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.410+0000 7fadd5684640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fadbc0a3c30 0x7fadbc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.410+0000 7fadc67fc640 1 -- 172.21.15.72:0/21071546 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fadc80f1840 con 0x7fadd0102a00 2024-04-25T13:11:09.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.411+0000 7fadd5684640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fadbc0a3c30 0x7fadbc0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fadc00058c0 tx=0x7fadc0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:09.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.415+0000 7fadc67fc640 1 -- 172.21.15.72:0/21071546 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadc80be370 con 0x7fadd0102a00 2024-04-25T13:11:09.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:09 smithi164 ceph-mon[32418]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T13:11:09.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:09 smithi164 ceph-mon[32418]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T13:11:09.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:09 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2738224329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:11:09.558 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:11:09 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b[62256]: 2024-04-25T13:11:09.098+0000 7f1645539700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:11:09.558 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:11:09 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b[62256]: 2024-04-25T13:11:09.098+0000 7f1645539700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T13:11:09.619 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.619+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fadd006ad90 con 0x7fadbc0a3c30 2024-04-25T13:11:09.624 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:09 smithi072 ceph-mon[29199]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T13:11:09.625 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:09 smithi072 ceph-mon[29199]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T13:11:09.625 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:09 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2738224329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:11:09.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.645+0000 7fadc67fc640 1 -- 172.21.15.72:0/21071546 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fadd006ad90 con 0x7fadbc0a3c30 2024-04-25T13:11:09.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fadbc0a3c30 msgr2=0x7fadbc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fadbc0a3c30 0x7fadbc0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fadc00058c0 tx=0x7fadc0005850 comp rx=0 tx=0).stop 2024-04-25T13:11:09.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 msgr2=0x7fadd019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 0x7fadd019e660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fadc80097b0 tx=0x7fadc8031840 comp rx=0 tx=0).stop 2024-04-25T13:11:09.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 shutdown_connections 2024-04-25T13:11:09.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fadbc0a3c30 0x7fadbc0a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadd0102a00 0x7fadd019e660 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0102040 0x7fadd01975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 --2- 172.21.15.72:0/21071546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0100e60 0x7fadd01970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 >> 172.21.15.72:0/21071546 conn(0x7fadd00fc440 msgr2=0x7fadd0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:09.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 shutdown_connections 2024-04-25T13:11:09.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.647+0000 7fadd78e8640 1 -- 172.21.15.72:0/21071546 wait complete. 2024-04-25T13:11:09.658 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T13:11:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:09 smithi193 ceph-mon[32743]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T13:11:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:09 smithi193 ceph-mon[32743]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T13:11:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:09 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2738224329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:11:09.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.760+0000 7f8903508640 1 -- 172.21.15.72:0/3955909798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 msgr2=0x7f88fc108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.760+0000 7f8903508640 1 --2- 172.21.15.72:0/3955909798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 0x7f88fc108a60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009ca0 tx=0x7f88f402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 -- 172.21.15.72:0/3955909798 shutdown_connections 2024-04-25T13:11:09.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 --2- 172.21.15.72:0/3955909798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 0x7f88fc108a60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 --2- 172.21.15.72:0/3955909798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc101810 0x7f88fc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 --2- 172.21.15.72:0/3955909798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc100620 0x7f88fc100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 -- 172.21.15.72:0/3955909798 >> 172.21.15.72:0/3955909798 conn(0x7f88fc0fbb90 msgr2=0x7f88fc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 -- 172.21.15.72:0/3955909798 shutdown_connections 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 -- 172.21.15.72:0/3955909798 wait complete. 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 Processor -- start 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 -- start start 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc100620 0x7f88fc197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc101810 0x7f88fc1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 0x7f88fc19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc10c130 con 0x7f88fc1021d0 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc10bfb0 con 0x7f88fc101810 2024-04-25T13:11:09.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8903508640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc10c2b0 con 0x7f88fc100620 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8901aa5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 0x7f88fc19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8901aa5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 0x7f88fc19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56334/0 (socket says 172.21.15.72:56334) 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8901aa5640 1 -- 172.21.15.72:0/960861996 learned_addr learned my addr 172.21.15.72:0/960861996 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.761+0000 7f8900aa3640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc101810 0x7f88fc1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f89012a4640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc100620 0x7f88fc197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f8901aa5640 1 -- 172.21.15.72:0/960861996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc100620 msgr2=0x7f88fc197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f8901aa5640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc100620 0x7f88fc197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f8901aa5640 1 -- 172.21.15.72:0/960861996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc101810 msgr2=0x7f88fc1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f8901aa5640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc101810 0x7f88fc1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f8901aa5640 1 -- 172.21.15.72:0/960861996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc19ec40 con 0x7f88fc1021d0 2024-04-25T13:11:09.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f8900aa3640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc101810 0x7f88fc1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:11:09.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f8901aa5640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 0x7f88fc19e5d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f88f40097b0 tx=0x7f88f4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:09.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f88f27fc640 1 -- 172.21.15.72:0/960861996 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4003bf0 con 0x7f88fc1021d0 2024-04-25T13:11:09.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f88f27fc640 1 -- 172.21.15.72:0/960861996 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f4003d90 con 0x7f88fc1021d0 2024-04-25T13:11:09.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f88f27fc640 1 -- 172.21.15.72:0/960861996 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f402fc70 con 0x7f88fc1021d0 2024-04-25T13:11:09.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88fc19eed0 con 0x7f88fc1021d0 2024-04-25T13:11:09.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.762+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88fc19f340 con 0x7f88fc1021d0 2024-04-25T13:11:09.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.763+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88fc108340 con 0x7f88fc1021d0 2024-04-25T13:11:09.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.764+0000 7f88f27fc640 1 -- 172.21.15.72:0/960861996 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f88f4004040 con 0x7f88fc1021d0 2024-04-25T13:11:09.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.765+0000 7f88f27fc640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f88e80a3bb0 0x7f88e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.765+0000 7f89012a4640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f88e80a3bb0 0x7f88e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.765+0000 7f88f27fc640 1 -- 172.21.15.72:0/960861996 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f88f40f18e0 con 0x7f88fc1021d0 2024-04-25T13:11:09.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.765+0000 7f89012a4640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f88e80a3bb0 0x7f88e80a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f88f80058c0 tx=0x7f88f8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:09.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.770+0000 7f88f27fc640 1 -- 172.21.15.72:0/960861996 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88f40be410 con 0x7f88fc1021d0 2024-04-25T13:11:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:09.976+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f88fc062390 con 0x7f88e80a3bb0 2024-04-25T13:11:10.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.005+0000 7f88f27fc640 1 -- 172.21.15.72:0/960861996 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f88fc062390 con 0x7f88e80a3bb0 2024-04-25T13:11:10.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f88e80a3bb0 msgr2=0x7f88e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f88e80a3bb0 0x7f88e80a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f88f80058c0 tx=0x7f88f8005850 comp rx=0 tx=0).stop 2024-04-25T13:11:10.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 msgr2=0x7f88fc19e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 0x7f88fc19e5d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f88f40097b0 tx=0x7f88f4031840 comp rx=0 tx=0).stop 2024-04-25T13:11:10.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 shutdown_connections 2024-04-25T13:11:10.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f88e80a3bb0 0x7f88e80a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88fc1021d0 0x7f88fc19e5d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc101810 0x7f88fc1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 --2- 172.21.15.72:0/960861996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc100620 0x7f88fc197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 >> 172.21.15.72:0/960861996 conn(0x7f88fc0fbb90 msgr2=0x7f88fc0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:10.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 shutdown_connections 2024-04-25T13:11:10.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.007+0000 7f8903508640 1 -- 172.21.15.72:0/960861996 wait complete. 2024-04-25T13:11:10.017 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:11:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:11:09.624+0000 7f558cc5d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:11:10.017 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:11:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:11:09.624+0000 7f558cc5d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:11:10.018 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:11:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:11:09.624+0000 7f558cc5d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:11:10.018 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:11:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:11:09.624+0000 7f558cc5d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:11:10.018 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:11:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:11:09.624+0000 7f558cc5d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:11:10.019 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T13:11:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 -- 172.21.15.72:0/3157648151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102a00 msgr2=0x7fe5f4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 --2- 172.21.15.72:0/3157648151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102a00 0x7fe5f4109320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009ca0 tx=0x7fe5ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 -- 172.21.15.72:0/3157648151 shutdown_connections 2024-04-25T13:11:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 --2- 172.21.15.72:0/3157648151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102a00 0x7fe5f4109320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 --2- 172.21.15.72:0/3157648151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f4102040 0x7fe5f41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 --2- 172.21.15.72:0/3157648151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5f4100e60 0x7fe5f4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 -- 172.21.15.72:0/3157648151 >> 172.21.15.72:0/3157648151 conn(0x7fe5f40fc440 msgr2=0x7fe5f40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 -- 172.21.15.72:0/3157648151 shutdown_connections 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 -- 172.21.15.72:0/3157648151 wait complete. 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 Processor -- start 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 -- start start 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f4100e60 0x7fe5f406b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102040 0x7fe5f419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5f4102a00 0x7fe5f419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f410ca70 con 0x7fe5f4102040 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f410c8f0 con 0x7fe5f4100e60 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.121+0000 7fe5faa85640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f410cbf0 con 0x7fe5f4102a00 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102040 0x7fe5f419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102040 0x7fe5f419cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56346/0 (socket says 172.21.15.72:56346) 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f3fff640 1 -- 172.21.15.72:0/4152654795 learned_addr learned my addr 172.21.15.72:0/4152654795 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f8821640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f4100e60 0x7fe5f406b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f9022640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5f4102a00 0x7fe5f419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f3fff640 1 -- 172.21.15.72:0/4152654795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5f4102a00 msgr2=0x7fe5f419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f3fff640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5f4102a00 0x7fe5f419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f3fff640 1 -- 172.21.15.72:0/4152654795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f4100e60 msgr2=0x7fe5f406b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f3fff640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f4100e60 0x7fe5f406b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f3fff640 1 -- 172.21.15.72:0/4152654795 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f419fe90 con 0x7fe5f4102040 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f3fff640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102040 0x7fe5f419cf60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000d950 tx=0x7fe5e000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f1ffb640 1 -- 172.21.15.72:0/4152654795 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0017030 con 0x7fe5f4102040 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f1ffb640 1 -- 172.21.15.72:0/4152654795 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e0004020 con 0x7fe5f4102040 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f41a0120 con 0x7fe5f4102040 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5f1ffb640 1 -- 172.21.15.72:0/4152654795 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0005560 con 0x7fe5f4102040 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.122+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f41a06c0 con 0x7fe5f4102040 2024-04-25T13:11:10.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.123+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f4108200 con 0x7fe5f4102040 2024-04-25T13:11:10.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.124+0000 7fe5f1ffb640 1 -- 172.21.15.72:0/4152654795 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe5e0009a30 con 0x7fe5f4102040 2024-04-25T13:11:10.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.125+0000 7fe5f1ffb640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe5e40a3c30 0x7fe5e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.125+0000 7fe5f8821640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe5e40a3c30 0x7fe5e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.125+0000 7fe5f1ffb640 1 -- 172.21.15.72:0/4152654795 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fe5e00cfc40 con 0x7fe5f4102040 2024-04-25T13:11:10.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.125+0000 7fe5f8821640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe5e40a3c30 0x7fe5e40a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4101bb0 tx=0x7fe5e8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.129+0000 7fe5f1ffb640 1 -- 172.21.15.72:0/4152654795 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5e009c770 con 0x7fe5f4102040 2024-04-25T13:11:10.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.337+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fe5f4062420 con 0x7fe5e40a3c30 2024-04-25T13:11:10.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.343+0000 7fe5f1ffb640 1 -- 172.21.15.72:0/4152654795 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe5f4062420 con 0x7fe5e40a3c30 2024-04-25T13:11:10.343 INFO:teuthology.orchestra.run.smithi072.stdout:/volumes/qa/sv_0/0f068763-aa56-4d05-a13a-669e8fe71eec 2024-04-25T13:11:10.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe5e40a3c30 msgr2=0x7fe5e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe5e40a3c30 0x7fe5e40a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4101bb0 tx=0x7fe5e8009110 comp rx=0 tx=0).stop 2024-04-25T13:11:10.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102040 msgr2=0x7fe5f419cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102040 0x7fe5f419cf60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000d950 tx=0x7fe5e000dd50 comp rx=0 tx=0).stop 2024-04-25T13:11:10.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 shutdown_connections 2024-04-25T13:11:10.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe5e40a3c30 0x7fe5e40a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5f4102a00 0x7fe5f419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f4102040 0x7fe5f419cf60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 --2- 172.21.15.72:0/4152654795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f4100e60 0x7fe5f406b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 >> 172.21.15.72:0/4152654795 conn(0x7fe5f40fc440 msgr2=0x7fe5f41053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:10.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 shutdown_connections 2024-04-25T13:11:10.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.345+0000 7fe5faa85640 1 -- 172.21.15.72:0/4152654795 wait complete. 2024-04-25T13:11:10.356 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T13:11:10.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.459+0000 7f885314f640 1 -- 172.21.15.72:0/2879694089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103bf0 msgr2=0x7f884c10a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.459+0000 7f885314f640 1 --2- 172.21.15.72:0/2879694089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103bf0 0x7f884c10a480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8844009cd0 tx=0x7f884402a8e0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.459+0000 7f885314f640 1 -- 172.21.15.72:0/2879694089 shutdown_connections 2024-04-25T13:11:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.459+0000 7f885314f640 1 --2- 172.21.15.72:0/2879694089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103bf0 0x7f884c10a480 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.459+0000 7f885314f640 1 --2- 172.21.15.72:0/2879694089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c103230 0x7f884c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.459+0000 7f885314f640 1 --2- 172.21.15.72:0/2879694089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f884c100260 0x7f884c100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.459+0000 7f885314f640 1 -- 172.21.15.72:0/2879694089 >> 172.21.15.72:0/2879694089 conn(0x7f884c0fbb90 msgr2=0x7f884c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.459+0000 7f885314f640 1 -- 172.21.15.72:0/2879694089 shutdown_connections 2024-04-25T13:11:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.459+0000 7f885314f640 1 -- 172.21.15.72:0/2879694089 wait complete. 2024-04-25T13:11:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f885314f640 1 Processor -- start 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f885314f640 1 -- start start 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f885314f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c100260 0x7f884c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f885314f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103230 0x7f884c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f885314f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f884c103bf0 0x7f884c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f885314f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f884c10dac0 con 0x7f884c103230 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f885314f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f884c10d940 con 0x7f884c103bf0 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f885314f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f884c10dc40 con 0x7f884c100260 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f884bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103230 0x7f884c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f884bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103230 0x7f884c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56352/0 (socket says 172.21.15.72:56352) 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f884bfff640 1 -- 172.21.15.72:0/3549233066 learned_addr learned my addr 172.21.15.72:0/3549233066 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f8850eeb640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c100260 0x7f884c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f88516ec640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f884c103bf0 0x7f884c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f884bfff640 1 -- 172.21.15.72:0/3549233066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c100260 msgr2=0x7f884c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f884bfff640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c100260 0x7f884c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f884bfff640 1 -- 172.21.15.72:0/3549233066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f884c103bf0 msgr2=0x7f884c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f884bfff640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f884c103bf0 0x7f884c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f884bfff640 1 -- 172.21.15.72:0/3549233066 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c19fdc0 con 0x7f884c103230 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f88516ec640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f884c103bf0 0x7f884c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.460+0000 7f884bfff640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103230 0x7f884c1986e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f883800d950 tx=0x7f883800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.461+0000 7f8849ffb640 1 -- 172.21.15.72:0/3549233066 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838017070 con 0x7f884c103230 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.461+0000 7f8849ffb640 1 -- 172.21.15.72:0/3549233066 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8838009c30 con 0x7f884c103230 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.461+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c1a00b0 con 0x7f884c103230 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.461+0000 7f8849ffb640 1 -- 172.21.15.72:0/3549233066 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883800f620 con 0x7f884c103230 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.461+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c1a0600 con 0x7f884c103230 2024-04-25T13:11:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.462+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c109380 con 0x7f884c103230 2024-04-25T13:11:10.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.463+0000 7f8849ffb640 1 -- 172.21.15.72:0/3549233066 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8838010460 con 0x7f884c103230 2024-04-25T13:11:10.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.463+0000 7f8849ffb640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f883c0a3bb0 0x7f883c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.463+0000 7f8850eeb640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f883c0a3bb0 0x7f883c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.463+0000 7f8849ffb640 1 -- 172.21.15.72:0/3549233066 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f88380cff90 con 0x7f884c103230 2024-04-25T13:11:10.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.464+0000 7f8850eeb640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f883c0a3bb0 0x7f883c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8840005fd0 tx=0x7f8840005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:10.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.468+0000 7f8849ffb640 1 -- 172.21.15.72:0/3549233066 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f883809cac0 con 0x7f884c103230 2024-04-25T13:11:10.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:10 smithi164 ceph-mon[32418]: pgmap v289: 97 pgs: 79 active+clean, 18 creating+peering; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 5 op/s 2024-04-25T13:11:10.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:10 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2738224329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:11:10.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:10 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] up:active 2024-04-25T13:11:10.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:10 smithi164 ceph-mon[32418]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T13:11:10.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:10 smithi164 ceph-mon[32418]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:11:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.672+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f884c062420 con 0x7f883c0a3bb0 2024-04-25T13:11:10.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.698+0000 7f8849ffb640 1 -- 172.21.15.72:0/3549233066 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f884c062420 con 0x7f883c0a3bb0 2024-04-25T13:11:10.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f883c0a3bb0 msgr2=0x7f883c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f883c0a3bb0 0x7f883c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8840005fd0 tx=0x7f8840005850 comp rx=0 tx=0).stop 2024-04-25T13:11:10.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103230 msgr2=0x7f884c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103230 0x7f884c1986e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f883800d950 tx=0x7f883800de10 comp rx=0 tx=0).stop 2024-04-25T13:11:10.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 shutdown_connections 2024-04-25T13:11:10.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f883c0a3bb0 0x7f883c0a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f884c103bf0 0x7f884c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f884c103230 0x7f884c1986e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 --2- 172.21.15.72:0/3549233066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c100260 0x7f884c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 >> 172.21.15.72:0/3549233066 conn(0x7f884c0fbb90 msgr2=0x7f884c106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:10.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 shutdown_connections 2024-04-25T13:11:10.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.700+0000 7f885314f640 1 -- 172.21.15.72:0/3549233066 wait complete. 2024-04-25T13:11:10.710 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:10 smithi072 ceph-mon[29199]: pgmap v289: 97 pgs: 79 active+clean, 18 creating+peering; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 5 op/s 2024-04-25T13:11:10.710 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:10 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2738224329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:11:10.710 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:10 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] up:active 2024-04-25T13:11:10.710 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:10 smithi072 ceph-mon[29199]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T13:11:10.710 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:10 smithi072 ceph-mon[29199]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:11:10.711 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T13:11:10.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:10 smithi193 ceph-mon[32743]: pgmap v289: 97 pgs: 79 active+clean, 18 creating+peering; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 5 op/s 2024-04-25T13:11:10.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:10 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2738224329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:11:10.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:10 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] up:active 2024-04-25T13:11:10.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:10 smithi193 ceph-mon[32743]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T13:11:10.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:10 smithi193 ceph-mon[32743]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.813+0000 7f5d2d810640 1 -- 172.21.15.72:0/1449400891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28102a70 msgr2=0x7f5d28104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.813+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1449400891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28102a70 0x7f5d28104e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009ca0 tx=0x7f5d1802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.813+0000 7f5d2d810640 1 -- 172.21.15.72:0/1449400891 shutdown_connections 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.813+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1449400891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d28107cd0 0x7f5d2810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.813+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1449400891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d281053a0 0x7f5d28107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.813+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1449400891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28102a70 0x7f5d28104e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.813+0000 7f5d2d810640 1 -- 172.21.15.72:0/1449400891 >> 172.21.15.72:0/1449400891 conn(0x7f5d280fc400 msgr2=0x7f5d280fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.813+0000 7f5d2d810640 1 -- 172.21.15.72:0/1449400891 shutdown_connections 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.813+0000 7f5d2d810640 1 -- 172.21.15.72:0/1449400891 wait complete. 2024-04-25T13:11:10.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d2d810640 1 Processor -- start 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d2d810640 1 -- start start 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d2d810640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d28102a70 0x7f5d281982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d2d810640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d281053a0 0x7f5d28198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d2d810640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d28107cd0 0x7f5d2819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d2d810640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2810db40 con 0x7f5d281053a0 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d2d810640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2810d9c0 con 0x7f5d28102a70 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d2d810640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2810dcc0 con 0x7f5d28107cd0 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d267fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d281053a0 0x7f5d28198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d267fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d281053a0 0x7f5d28198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56362/0 (socket says 172.21.15.72:56362) 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.814+0000 7f5d267fc640 1 -- 172.21.15.72:0/1551798724 learned_addr learned my addr 172.21.15.72:0/1551798724 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d26ffd640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d28102a70 0x7f5d281982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d277fe640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d28107cd0 0x7f5d2819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d267fc640 1 -- 172.21.15.72:0/1551798724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d28107cd0 msgr2=0x7f5d2819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d267fc640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d28107cd0 0x7f5d2819f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d267fc640 1 -- 172.21.15.72:0/1551798724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d28102a70 msgr2=0x7f5d281982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d267fc640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d28102a70 0x7f5d281982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d267fc640 1 -- 172.21.15.72:0/1551798724 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2819fef0 con 0x7f5d281053a0 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d26ffd640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d28102a70 0x7f5d281982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d267fc640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d281053a0 0x7f5d28198810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00d770 tx=0x7f5d1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d2c80e640 1 -- 172.21.15.72:0/1551798724 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c017070 con 0x7f5d281053a0 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d2c80e640 1 -- 172.21.15.72:0/1551798724 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1c009c30 con 0x7f5d281053a0 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d281a01e0 con 0x7f5d281053a0 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d2c80e640 1 -- 172.21.15.72:0/1551798724 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00f620 con 0x7f5d281053a0 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.815+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d281a07b0 con 0x7f5d281053a0 2024-04-25T13:11:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.816+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d280fdef0 con 0x7f5d281053a0 2024-04-25T13:11:10.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.818+0000 7f5d2c80e640 1 -- 172.21.15.72:0/1551798724 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d1c01a070 con 0x7f5d281053a0 2024-04-25T13:11:10.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.818+0000 7f5d2c80e640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d100a3bb0 0x7f5d100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:10.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.818+0000 7f5d26ffd640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d100a3bb0 0x7f5d100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:10.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.818+0000 7f5d2c80e640 1 -- 172.21.15.72:0/1551798724 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5d1c0cfe60 con 0x7f5d281053a0 2024-04-25T13:11:10.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.819+0000 7f5d26ffd640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d100a3bb0 0x7f5d100a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d18031b50 tx=0x7f5d18005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:10.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:10.822+0000 7f5d2c80e640 1 -- 172.21.15.72:0/1551798724 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d1c009de0 con 0x7f5d281053a0 2024-04-25T13:11:11.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.032+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5d28062420 con 0x7f5d100a3bb0 2024-04-25T13:11:11.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.037+0000 7f5d2c80e640 1 -- 172.21.15.72:0/1551798724 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5d28062420 con 0x7f5d100a3bb0 2024-04-25T13:11:11.037 INFO:teuthology.orchestra.run.smithi072.stdout:/volumes/qa/sv_1/b84445af-f374-4c76-ad5e-e65b51e37dce 2024-04-25T13:11:11.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d100a3bb0 msgr2=0x7f5d100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d100a3bb0 0x7f5d100a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d18031b50 tx=0x7f5d18005ef0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d281053a0 msgr2=0x7f5d28198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d281053a0 0x7f5d28198810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00d770 tx=0x7f5d1c00dc30 comp rx=0 tx=0).stop 2024-04-25T13:11:11.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 shutdown_connections 2024-04-25T13:11:11.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d100a3bb0 0x7f5d100a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d28107cd0 0x7f5d2819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d281053a0 0x7f5d28198810 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 --2- 172.21.15.72:0/1551798724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d28102a70 0x7f5d281982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 >> 172.21.15.72:0/1551798724 conn(0x7f5d280fc400 msgr2=0x7f5d281033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:11.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 shutdown_connections 2024-04-25T13:11:11.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.039+0000 7f5d2d810640 1 -- 172.21.15.72:0/1551798724 wait complete. 2024-04-25T13:11:11.050 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:11:11.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.148+0000 7f6bea86b640 1 -- 172.21.15.72:0/701359835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4103c00 msgr2=0x7f6be410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.148+0000 7f6bea86b640 1 --2- 172.21.15.72:0/701359835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4103c00 0x7f6be410a490 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009cd0 tx=0x7f6bdc02a8e0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.148+0000 7f6bea86b640 1 -- 172.21.15.72:0/701359835 shutdown_connections 2024-04-25T13:11:11.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.148+0000 7f6bea86b640 1 --2- 172.21.15.72:0/701359835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4103c00 0x7f6be410a490 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.148+0000 7f6bea86b640 1 --2- 172.21.15.72:0/701359835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6be4103240 0x7f6be41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.148+0000 7f6bea86b640 1 --2- 172.21.15.72:0/701359835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6be4100270 0x7f6be4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.148+0000 7f6bea86b640 1 -- 172.21.15.72:0/701359835 >> 172.21.15.72:0/701359835 conn(0x7f6be40fbbc0 msgr2=0x7f6be40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.148+0000 7f6bea86b640 1 -- 172.21.15.72:0/701359835 shutdown_connections 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.148+0000 7f6bea86b640 1 -- 172.21.15.72:0/701359835 wait complete. 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6bea86b640 1 Processor -- start 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6bea86b640 1 -- start start 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6bea86b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4100270 0x7f6be4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6bea86b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6be4103240 0x7f6be4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6bea86b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6be4103c00 0x7f6be419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6bea86b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6be410dad0 con 0x7f6be4100270 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6bea86b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6be410d950 con 0x7f6be4103240 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6bea86b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6be410dc50 con 0x7f6be4103c00 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4100270 0x7f6be4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4100270 0x7f6be4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56384/0 (socket says 172.21.15.72:56384) 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6be3fff640 1 -- 172.21.15.72:0/1788884763 learned_addr learned my addr 172.21.15.72:0/1788884763 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:11.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6be3fff640 1 -- 172.21.15.72:0/1788884763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6be4103c00 msgr2=0x7f6be419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6be3fff640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6be4103c00 0x7f6be419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6be3fff640 1 -- 172.21.15.72:0/1788884763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6be4103240 msgr2=0x7f6be4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6be3fff640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6be4103240 0x7f6be4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6be3fff640 1 -- 172.21.15.72:0/1788884763 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be419fd30 con 0x7f6be4100270 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.149+0000 7f6be3fff640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4100270 0x7f6be4198110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800c970 tx=0x7f6bd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.150+0000 7f6be17fa640 1 -- 172.21.15.72:0/1788884763 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8007c10 con 0x7f6be4100270 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.150+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be419ffc0 con 0x7f6be4100270 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.150+0000 7f6be17fa640 1 -- 172.21.15.72:0/1788884763 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd8007db0 con 0x7f6be4100270 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.150+0000 7f6be17fa640 1 -- 172.21.15.72:0/1788884763 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800f7d0 con 0x7f6be4100270 2024-04-25T13:11:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.150+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be41a04e0 con 0x7f6be4100270 2024-04-25T13:11:11.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.151+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be4109370 con 0x7f6be4100270 2024-04-25T13:11:11.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.152+0000 7f6be17fa640 1 -- 172.21.15.72:0/1788884763 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6bd8018480 con 0x7f6be4100270 2024-04-25T13:11:11.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.152+0000 7f6be17fa640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6bcc0a3bb0 0x7f6bcc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.152+0000 7f6be37fe640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6bcc0a3bb0 0x7f6bcc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.152+0000 7f6be17fa640 1 -- 172.21.15.72:0/1788884763 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f6bd80d0690 con 0x7f6be4100270 2024-04-25T13:11:11.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.153+0000 7f6be37fe640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6bcc0a3bb0 0x7f6bcc0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005fd0 tx=0x7f6bd4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:11.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.157+0000 7f6be17fa640 1 -- 172.21.15.72:0/1788884763 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bd809d250 con 0x7f6be4100270 2024-04-25T13:11:11.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.307+0000 7f6be17fa640 1 -- 172.21.15.72:0/1788884763 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6bd80991c0 con 0x7f6be4100270 2024-04-25T13:11:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.379+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6be402d7b0 con 0x7f6be4100270 2024-04-25T13:11:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.379+0000 7f6be17fa640 1 -- 172.21.15.72:0/1788884763 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11881 (secure 0 0 0) 0x7f6bd80cc100 con 0x7f6be4100270 2024-04-25T13:11:11.380 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:11.380 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:11:04.601639+0000","modified":"2024-04-25T13:11:09.292132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24493,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6839/1444196308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1444196308},{"type":"v1","addr":"172.21.15.164:6839","nonce":1444196308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24478,24493,24509]},"id":1}]} 2024-04-25T13:11:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 20 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.381+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6bcc0a3bb0 msgr2=0x7f6bcc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.381+0000 7f6bea86b640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6bcc0a3bb0 0x7f6bcc0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005fd0 tx=0x7f6bd4005850 comp rx=0 tx=0).stop 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.381+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4100270 msgr2=0x7f6be4198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.381+0000 7f6bea86b640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4100270 0x7f6be4198110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800c970 tx=0x7f6bd800ce30 comp rx=0 tx=0).stop 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.382+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 shutdown_connections 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.382+0000 7f6bea86b640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6bcc0a3bb0 0x7f6bcc0a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.382+0000 7f6bea86b640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6be4103c00 0x7f6be419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.382+0000 7f6bea86b640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6be4103240 0x7f6be4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.382+0000 7f6bea86b640 1 --2- 172.21.15.72:0/1788884763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6be4100270 0x7f6be4198110 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.382+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 >> 172.21.15.72:0/1788884763 conn(0x7f6be40fbbc0 msgr2=0x7f6be4106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:11.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.382+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 shutdown_connections 2024-04-25T13:11:11.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.382+0000 7f6bea86b640 1 -- 172.21.15.72:0/1788884763 wait complete. 2024-04-25T13:11:11.393 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:11:11.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 -- 172.21.15.72:0/2538714479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638107c70 msgr2=0x7ff63810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/2538714479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638107c70 0x7ff63810a0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff630009ca0 tx=0x7ff63002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 -- 172.21.15.72:0/2538714479 shutdown_connections 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/2538714479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638107c70 0x7ff63810a0c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/2538714479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff638105340 0x7ff638107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/2538714479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff638102a10 0x7ff638104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 -- 172.21.15.72:0/2538714479 >> 172.21.15.72:0/2538714479 conn(0x7ff6380fc400 msgr2=0x7ff6380fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 -- 172.21.15.72:0/2538714479 shutdown_connections 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 -- 172.21.15.72:0/2538714479 wait complete. 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.490+0000 7ff63d5e5640 1 Processor -- start 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff63d5e5640 1 -- start start 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff63d5e5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff638102a10 0x7ff638198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff63d5e5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638105340 0x7ff6381986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff63d5e5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff638107c70 0x7ff63819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff63d5e5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff63810db40 con 0x7ff638105340 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff63d5e5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff63810d9c0 con 0x7ff638107c70 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff63d5e5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff63810dcc0 con 0x7ff638102a10 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6367fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638105340 0x7ff6381986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6367fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638105340 0x7ff6381986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56414/0 (socket says 172.21.15.72:56414) 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6367fc640 1 -- 172.21.15.72:0/665586143 learned_addr learned my addr 172.21.15.72:0/665586143 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6377fe640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff638107c70 0x7ff63819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6367fc640 1 -- 172.21.15.72:0/665586143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff638102a10 msgr2=0x7ff638198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff636ffd640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff638102a10 0x7ff638198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6367fc640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff638102a10 0x7ff638198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6367fc640 1 -- 172.21.15.72:0/665586143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff638107c70 msgr2=0x7ff63819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6367fc640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff638107c70 0x7ff63819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6367fc640 1 -- 172.21.15.72:0/665586143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63819fd80 con 0x7ff638105340 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6377fe640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff638107c70 0x7ff63819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff636ffd640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff638102a10 0x7ff638198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff6367fc640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638105340 0x7ff6381986a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00c970 tx=0x7ff62c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff61ffff640 1 -- 172.21.15.72:0/665586143 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c014070 con 0x7ff638105340 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6381a0070 con 0x7ff638105340 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.491+0000 7ff61ffff640 1 -- 172.21.15.72:0/665586143 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff62c004440 con 0x7ff638105340 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.492+0000 7ff61ffff640 1 -- 172.21.15.72:0/665586143 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c003e50 con 0x7ff638105340 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.492+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6381a0690 con 0x7ff638105340 2024-04-25T13:11:11.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.492+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6380fdef0 con 0x7ff638105340 2024-04-25T13:11:11.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.493+0000 7ff61ffff640 1 -- 172.21.15.72:0/665586143 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff62c017070 con 0x7ff638105340 2024-04-25T13:11:11.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.494+0000 7ff61ffff640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6200a3de0 0x7ff6200a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.494+0000 7ff61ffff640 1 -- 172.21.15.72:0/665586143 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff62c0d06f0 con 0x7ff638105340 2024-04-25T13:11:11.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.494+0000 7ff636ffd640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6200a3de0 0x7ff6200a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.494+0000 7ff636ffd640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6200a3de0 0x7ff6200a62a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff628006fd0 tx=0x7ff628008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:11.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.498+0000 7ff61ffff640 1 -- 172.21.15.72:0/665586143 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff62c01e020 con 0x7ff638105340 2024-04-25T13:11:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.675+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff63802d7e0 con 0x7ff638105340 2024-04-25T13:11:11.676 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.676+0000 7ff61ffff640 1 -- 172.21.15.72:0/665586143 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22243 (secure 0 0 0) 0x7ff62c09d120 con 0x7ff638105340 2024-04-25T13:11:11.677 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:11.677 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":88,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:11:06.264801+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:11:11.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6200a3de0 msgr2=0x7ff6200a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6200a3de0 0x7ff6200a62a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff628006fd0 tx=0x7ff628008040 comp rx=0 tx=0).stop 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638105340 msgr2=0x7ff6381986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638105340 0x7ff6381986a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00c970 tx=0x7ff62c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 shutdown_connections 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6200a3de0 0x7ff6200a62a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff638107c70 0x7ff63819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff638105340 0x7ff6381986a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 --2- 172.21.15.72:0/665586143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff638102a10 0x7ff638198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.678+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 >> 172.21.15.72:0/665586143 conn(0x7ff6380fc400 msgr2=0x7ff638103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.679+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 shutdown_connections 2024-04-25T13:11:11.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.679+0000 7ff63d5e5640 1 -- 172.21.15.72:0/665586143 wait complete. 2024-04-25T13:11:11.688 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:11 smithi072 ceph-mon[29199]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:11.688 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:11 smithi072 ceph-mon[29199]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:11.689 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:11 smithi072 ceph-mon[29199]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:11.690 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:11:11.691 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:11:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:11 smithi193 ceph-mon[32743]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:11 smithi193 ceph-mon[32743]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:11 smithi193 ceph-mon[32743]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:11.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.788+0000 7f6586a91640 1 -- 172.21.15.72:0/2465521368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580101fe0 msgr2=0x7f6580102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.788+0000 7f6586a91640 1 --2- 172.21.15.72:0/2465521368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580101fe0 0x7f6580102460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6568009900 tx=0x7f656802a410 comp rx=0 tx=0).stop 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.788+0000 7f6586a91640 1 -- 172.21.15.72:0/2465521368 shutdown_connections 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.788+0000 7f6586a91640 1 --2- 172.21.15.72:0/2465521368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65801029a0 0x7f65801092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.788+0000 7f6586a91640 1 --2- 172.21.15.72:0/2465521368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580101fe0 0x7f6580102460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.788+0000 7f6586a91640 1 --2- 172.21.15.72:0/2465521368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6580100e90 0x7f65801012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.788+0000 7f6586a91640 1 -- 172.21.15.72:0/2465521368 >> 172.21.15.72:0/2465521368 conn(0x7f65800fc400 msgr2=0x7f65800fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 -- 172.21.15.72:0/2465521368 shutdown_connections 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 -- 172.21.15.72:0/2465521368 wait complete. 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 Processor -- start 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 -- start start 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580100e90 0x7f65801982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6580101fe0 0x7f6580198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65801029a0 0x7f658019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f658010ca40 con 0x7f6580100e90 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f658010c8c0 con 0x7f6580101fe0 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f6586a91640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f658010cbc0 con 0x7f65801029a0 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f658482d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580100e90 0x7f65801982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f658482d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580100e90 0x7f65801982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56428/0 (socket says 172.21.15.72:56428) 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f658482d640 1 -- 172.21.15.72:0/2866290549 learned_addr learned my addr 172.21.15.72:0/2866290549 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.789+0000 7f658502e640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65801029a0 0x7f658019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f658482d640 1 -- 172.21.15.72:0/2866290549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65801029a0 msgr2=0x7f658019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f658482d640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65801029a0 0x7f658019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f658482d640 1 -- 172.21.15.72:0/2866290549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6580101fe0 msgr2=0x7f6580198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f658482d640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6580101fe0 0x7f6580198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f658482d640 1 -- 172.21.15.72:0/2866290549 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658019ff10 con 0x7f6580100e90 2024-04-25T13:11:11.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f658482d640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580100e90 0x7f65801982f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f657800ba50 tx=0x7f657800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:11.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f657dffb640 1 -- 172.21.15.72:0/2866290549 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578019070 con 0x7f6580100e90 2024-04-25T13:11:11.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f657dffb640 1 -- 172.21.15.72:0/2866290549 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657800cbe0 con 0x7f6580100e90 2024-04-25T13:11:11.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65801a0200 con 0x7f6580100e90 2024-04-25T13:11:11.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f657dffb640 1 -- 172.21.15.72:0/2866290549 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65780075d0 con 0x7f6580100e90 2024-04-25T13:11:11.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.790+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65801a07d0 con 0x7f6580100e90 2024-04-25T13:11:11.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.791+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65801a0a90 con 0x7f6580100e90 2024-04-25T13:11:11.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.792+0000 7f657dffb640 1 -- 172.21.15.72:0/2866290549 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f657801c070 con 0x7f6580100e90 2024-04-25T13:11:11.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.792+0000 7f657dffb640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f656c0a3c40 0x7f656c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.793+0000 7f657ffff640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f656c0a3c40 0x7f656c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.793+0000 7f657dffb640 1 -- 172.21.15.72:0/2866290549 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f65780cfa10 con 0x7f6580100e90 2024-04-25T13:11:11.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.793+0000 7f657ffff640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f656c0a3c40 0x7f656c0a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65680096c0 tx=0x7f6568031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:11.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.797+0000 7f657dffb640 1 -- 172.21.15.72:0/2866290549 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f657809c490 con 0x7f6580100e90 2024-04-25T13:11:11.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:11 smithi164 ceph-mon[32418]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:11.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:11 smithi164 ceph-mon[32418]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:11.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:11 smithi164 ceph-mon[32418]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:11.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.971+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65800fe650 con 0x7f656c0a3c40 2024-04-25T13:11:11.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.978+0000 7f657dffb640 1 -- 172.21.15.72:0/2866290549 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258751 (secure 0 0 0) 0x7f65800fe650 con 0x7f656c0a3c40 2024-04-25T13:11:11.981 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:11.982 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-25T13:11:11.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.981+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f656c0a3c40 msgr2=0x7f656c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.981+0000 7f6586a91640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f656c0a3c40 0x7f656c0a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65680096c0 tx=0x7f6568031400 comp rx=0 tx=0).stop 2024-04-25T13:11:11.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.981+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580100e90 msgr2=0x7f65801982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.981+0000 7f6586a91640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580100e90 0x7f65801982f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f657800ba50 tx=0x7f657800bf10 comp rx=0 tx=0).stop 2024-04-25T13:11:11.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.981+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 shutdown_connections 2024-04-25T13:11:11.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.981+0000 7f6586a91640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f656c0a3c40 0x7f656c0a6100 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.981+0000 7f6586a91640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65801029a0 0x7f658019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.981+0000 7f6586a91640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6580101fe0 0x7f6580198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.981+0000 7f6586a91640 1 --2- 172.21.15.72:0/2866290549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6580100e90 0x7f65801982f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.982+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 >> 172.21.15.72:0/2866290549 conn(0x7f65800fc400 msgr2=0x7f65800fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:11.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.982+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 shutdown_connections 2024-04-25T13:11:11.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:11.982+0000 7f6586a91640 1 -- 172.21.15.72:0/2866290549 wait complete. 2024-04-25T13:11:11.993 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-25T13:11:11.125635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594116,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":150,"num_read_kb":131,"num_write":279,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383012,"kb_used_data":13024,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558812,"statfs":{"total":1151940427776,"available":1151548223488,"internally_reserved":0,"allocated":13336576,"data_stored":7229538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3748,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":98,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.869077"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273029+0000","last_change":"2024-04-25T13:11:05.285490+0000","last_active":"2024-04-25T13:11:06.273029+0000","last_peered":"2024-04-25T13:11:06.273029+0000","last_clean":"2024-04-25T13:11:06.273029+0000","last_became_active":"2024-04-25T13:11:05.285066+0000","last_became_peered":"2024-04-25T13:11:05.285066+0000","last_unstale":"2024-04-25T13:11:06.273029+0000","last_undegraded":"2024-04-25T13:11:06.273029+0000","last_fullsized":"2024-04-25T13:11:06.273029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:30.002823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565459+0000","last_change":"2024-04-25T13:11:05.292670+0000","last_active":"2024-04-25T13:11:06.565459+0000","last_peered":"2024-04-25T13:11:06.565459+0000","last_clean":"2024-04-25T13:11:06.565459+0000","last_became_active":"2024-04-25T13:11:05.292279+0000","last_became_peered":"2024-04-25T13:11:05.292279+0000","last_unstale":"2024-04-25T13:11:06.565459+0000","last_undegraded":"2024-04-25T13:11:06.565459+0000","last_fullsized":"2024-04-25T13:11:06.565459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:26.439874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565225+0000","last_change":"2024-04-25T13:11:05.284228+0000","last_active":"2024-04-25T13:11:06.565225+0000","last_peered":"2024-04-25T13:11:06.565225+0000","last_clean":"2024-04-25T13:11:06.565225+0000","last_became_active":"2024-04-25T13:11:05.283877+0000","last_became_peered":"2024-04-25T13:11:05.283877+0000","last_unstale":"2024-04-25T13:11:06.565225+0000","last_undegraded":"2024-04-25T13:11:06.565225+0000","last_fullsized":"2024-04-25T13:11:06.565225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:19.768393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565866+0000","last_change":"2024-04-25T13:11:05.276516+0000","last_active":"2024-04-25T13:11:06.565866+0000","last_peered":"2024-04-25T13:11:06.565866+0000","last_clean":"2024-04-25T13:11:06.565866+0000","last_became_active":"2024-04-25T13:11:05.276174+0000","last_became_peered":"2024-04-25T13:11:05.276174+0000","last_unstale":"2024-04-25T13:11:06.565866+0000","last_undegraded":"2024-04-25T13:11:06.565866+0000","last_fullsized":"2024-04-25T13:11:06.565866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:27.801716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403554+0000","last_change":"2024-04-25T13:11:05.279010+0000","last_active":"2024-04-25T13:11:06.403554+0000","last_peered":"2024-04-25T13:11:06.403554+0000","last_clean":"2024-04-25T13:11:06.403554+0000","last_became_active":"2024-04-25T13:11:05.276879+0000","last_became_peered":"2024-04-25T13:11:05.276879+0000","last_unstale":"2024-04-25T13:11:06.403554+0000","last_undegraded":"2024-04-25T13:11:06.403554+0000","last_fullsized":"2024-04-25T13:11:06.403554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:14.191090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.292741+0000","last_change":"2024-04-25T13:11:05.291530+0000","last_active":"2024-04-25T13:11:05.292741+0000","last_peered":"2024-04-25T13:11:05.292741+0000","last_clean":"2024-04-25T13:11:05.292741+0000","last_became_active":"2024-04-25T13:11:05.291119+0000","last_became_peered":"2024-04-25T13:11:05.291119+0000","last_unstale":"2024-04-25T13:11:05.292741+0000","last_undegraded":"2024-04-25T13:11:05.292741+0000","last_fullsized":"2024-04-25T13:11:05.292741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:08.498624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272843+0000","last_change":"2024-04-25T13:11:05.281816+0000","last_active":"2024-04-25T13:11:06.272843+0000","last_peered":"2024-04-25T13:11:06.272843+0000","last_clean":"2024-04-25T13:11:06.272843+0000","last_became_active":"2024-04-25T13:11:05.281504+0000","last_became_peered":"2024-04-25T13:11:05.281504+0000","last_unstale":"2024-04-25T13:11:06.272843+0000","last_undegraded":"2024-04-25T13:11:06.272843+0000","last_fullsized":"2024-04-25T13:11:06.272843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.724238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564627+0000","last_change":"2024-04-25T13:11:05.279320+0000","last_active":"2024-04-25T13:11:06.564627+0000","last_peered":"2024-04-25T13:11:06.564627+0000","last_clean":"2024-04-25T13:11:06.564627+0000","last_became_active":"2024-04-25T13:11:05.278995+0000","last_became_peered":"2024-04-25T13:11:05.278995+0000","last_unstale":"2024-04-25T13:11:06.564627+0000","last_undegraded":"2024-04-25T13:11:06.564627+0000","last_fullsized":"2024-04-25T13:11:06.564627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:53.592529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565162+0000","last_change":"2024-04-25T13:11:05.279109+0000","last_active":"2024-04-25T13:11:06.565162+0000","last_peered":"2024-04-25T13:11:06.565162+0000","last_clean":"2024-04-25T13:11:06.565162+0000","last_became_active":"2024-04-25T13:11:05.278523+0000","last_became_peered":"2024-04-25T13:11:05.278523+0000","last_unstale":"2024-04-25T13:11:06.565162+0000","last_undegraded":"2024-04-25T13:11:06.565162+0000","last_fullsized":"2024-04-25T13:11:06.565162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:24.646025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.292953+0000","last_change":"2024-04-25T13:11:05.292039+0000","last_active":"2024-04-25T13:11:05.292953+0000","last_peered":"2024-04-25T13:11:05.292953+0000","last_clean":"2024-04-25T13:11:05.292953+0000","last_became_active":"2024-04-25T13:11:05.291649+0000","last_became_peered":"2024-04-25T13:11:05.291649+0000","last_unstale":"2024-04-25T13:11:05.292953+0000","last_undegraded":"2024-04-25T13:11:05.292953+0000","last_fullsized":"2024-04-25T13:11:05.292953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:35.392213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273124+0000","last_change":"2024-04-25T13:11:05.287067+0000","last_active":"2024-04-25T13:11:06.273124+0000","last_peered":"2024-04-25T13:11:06.273124+0000","last_clean":"2024-04-25T13:11:06.273124+0000","last_became_active":"2024-04-25T13:11:05.286664+0000","last_became_peered":"2024-04-25T13:11:05.286664+0000","last_unstale":"2024-04-25T13:11:06.273124+0000","last_undegraded":"2024-04-25T13:11:06.273124+0000","last_fullsized":"2024-04-25T13:11:06.273124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:12.677498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565670+0000","last_change":"2024-04-25T13:11:05.293192+0000","last_active":"2024-04-25T13:11:06.565670+0000","last_peered":"2024-04-25T13:11:06.565670+0000","last_clean":"2024-04-25T13:11:06.565670+0000","last_became_active":"2024-04-25T13:11:05.292663+0000","last_became_peered":"2024-04-25T13:11:05.292663+0000","last_unstale":"2024-04-25T13:11:06.565670+0000","last_undegraded":"2024-04-25T13:11:06.565670+0000","last_fullsized":"2024-04-25T13:11:06.565670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:14.942484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.566296+0000","last_change":"2024-04-25T13:11:05.278090+0000","last_active":"2024-04-25T13:11:06.566296+0000","last_peered":"2024-04-25T13:11:06.566296+0000","last_clean":"2024-04-25T13:11:06.566296+0000","last_became_active":"2024-04-25T13:11:05.277744+0000","last_became_peered":"2024-04-25T13:11:05.277744+0000","last_unstale":"2024-04-25T13:11:06.566296+0000","last_undegraded":"2024-04-25T13:11:06.566296+0000","last_fullsized":"2024-04-25T13:11:06.566296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:49.311241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564727+0000","last_change":"2024-04-25T13:11:05.277356+0000","last_active":"2024-04-25T13:11:06.564727+0000","last_peered":"2024-04-25T13:11:06.564727+0000","last_clean":"2024-04-25T13:11:06.564727+0000","last_became_active":"2024-04-25T13:11:05.276992+0000","last_became_peered":"2024-04-25T13:11:05.276992+0000","last_unstale":"2024-04-25T13:11:06.564727+0000","last_undegraded":"2024-04-25T13:11:06.564727+0000","last_fullsized":"2024-04-25T13:11:06.564727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:05.929613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564414+0000","last_change":"2024-04-25T13:11:05.271950+0000","last_active":"2024-04-25T13:11:06.564414+0000","last_peered":"2024-04-25T13:11:06.564414+0000","last_clean":"2024-04-25T13:11:06.564414+0000","last_became_active":"2024-04-25T13:11:05.271660+0000","last_became_peered":"2024-04-25T13:11:05.271660+0000","last_unstale":"2024-04-25T13:11:06.564414+0000","last_undegraded":"2024-04-25T13:11:06.564414+0000","last_fullsized":"2024-04-25T13:11:06.564414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:33.534854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564497+0000","last_change":"2024-04-25T13:11:05.283689+0000","last_active":"2024-04-25T13:11:06.564497+0000","last_peered":"2024-04-25T13:11:06.564497+0000","last_clean":"2024-04-25T13:11:06.564497+0000","last_became_active":"2024-04-25T13:11:05.283364+0000","last_became_peered":"2024-04-25T13:11:05.283364+0000","last_unstale":"2024-04-25T13:11:06.564497+0000","last_undegraded":"2024-04-25T13:11:06.564497+0000","last_fullsized":"2024-04-25T13:11:06.564497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:24.109350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273160+0000","last_change":"2024-04-25T13:11:05.280865+0000","last_active":"2024-04-25T13:11:06.273160+0000","last_peered":"2024-04-25T13:11:06.273160+0000","last_clean":"2024-04-25T13:11:06.273160+0000","last_became_active":"2024-04-25T13:11:05.280436+0000","last_became_peered":"2024-04-25T13:11:05.280436+0000","last_unstale":"2024-04-25T13:11:06.273160+0000","last_undegraded":"2024-04-25T13:11:06.273160+0000","last_fullsized":"2024-04-25T13:11:06.273160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:58.960578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272318+0000","last_change":"2024-04-25T13:11:05.282844+0000","last_active":"2024-04-25T13:11:06.272318+0000","last_peered":"2024-04-25T13:11:06.272318+0000","last_clean":"2024-04-25T13:11:06.272318+0000","last_became_active":"2024-04-25T13:11:05.282502+0000","last_became_peered":"2024-04-25T13:11:05.282502+0000","last_unstale":"2024-04-25T13:11:06.272318+0000","last_undegraded":"2024-04-25T13:11:06.272318+0000","last_fullsized":"2024-04-25T13:11:06.272318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:58.014902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564955+0000","last_change":"2024-04-25T13:11:05.276419+0000","last_active":"2024-04-25T13:11:06.564955+0000","last_peered":"2024-04-25T13:11:06.564955+0000","last_clean":"2024-04-25T13:11:06.564955+0000","last_became_active":"2024-04-25T13:11:05.275953+0000","last_became_peered":"2024-04-25T13:11:05.275953+0000","last_unstale":"2024-04-25T13:11:06.564955+0000","last_undegraded":"2024-04-25T13:11:06.564955+0000","last_fullsized":"2024-04-25T13:11:06.564955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:41.993632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.294580+0000","last_change":"2024-04-25T13:11:04.278345+0000","last_active":"2024-04-25T13:11:08.294580+0000","last_peered":"2024-04-25T13:11:08.294580+0000","last_clean":"2024-04-25T13:11:08.294580+0000","last_became_active":"2024-04-25T13:11:04.278017+0000","last_became_peered":"2024-04-25T13:11:04.278017+0000","last_unstale":"2024-04-25T13:11:08.294580+0000","last_undegraded":"2024-04-25T13:11:08.294580+0000","last_fullsized":"2024-04-25T13:11:08.294580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:34.496703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565116+0000","last_change":"2024-04-25T13:11:05.281069+0000","last_active":"2024-04-25T13:11:06.565116+0000","last_peered":"2024-04-25T13:11:06.565116+0000","last_clean":"2024-04-25T13:11:06.565116+0000","last_became_active":"2024-04-25T13:11:05.279428+0000","last_became_peered":"2024-04-25T13:11:05.279428+0000","last_unstale":"2024-04-25T13:11:06.565116+0000","last_undegraded":"2024-04-25T13:11:06.565116+0000","last_fullsized":"2024-04-25T13:11:06.565116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:53.224215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565213+0000","last_change":"2024-04-25T13:11:04.276017+0000","last_active":"2024-04-25T13:11:06.565213+0000","last_peered":"2024-04-25T13:11:06.565213+0000","last_clean":"2024-04-25T13:11:06.565213+0000","last_became_active":"2024-04-25T13:11:04.275718+0000","last_became_peered":"2024-04-25T13:11:04.275718+0000","last_unstale":"2024-04-25T13:11:06.565213+0000","last_undegraded":"2024-04-25T13:11:06.565213+0000","last_fullsized":"2024-04-25T13:11:06.565213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:45.850856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565503+0000","last_change":"2024-04-25T13:11:05.283185+0000","last_active":"2024-04-25T13:11:06.565503+0000","last_peered":"2024-04-25T13:11:06.565503+0000","last_clean":"2024-04-25T13:11:06.565503+0000","last_became_active":"2024-04-25T13:11:05.282876+0000","last_became_peered":"2024-04-25T13:11:05.282876+0000","last_unstale":"2024-04-25T13:11:06.565503+0000","last_undegraded":"2024-04-25T13:11:06.565503+0000","last_fullsized":"2024-04-25T13:11:06.565503+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:08.403124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.293012+0000","last_change":"2024-04-25T13:11:04.282188+0000","last_active":"2024-04-25T13:11:08.293012+0000","last_peered":"2024-04-25T13:11:08.293012+0000","last_clean":"2024-04-25T13:11:08.293012+0000","last_became_active":"2024-04-25T13:11:04.281569+0000","last_became_peered":"2024-04-25T13:11:04.281569+0000","last_unstale":"2024-04-25T13:11:08.293012+0000","last_undegraded":"2024-04-25T13:11:08.293012+0000","last_fullsized":"2024-04-25T13:11:08.293012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:45.502754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564612+0000","last_change":"2024-04-25T13:11:05.292858+0000","last_active":"2024-04-25T13:11:06.564612+0000","last_peered":"2024-04-25T13:11:06.564612+0000","last_clean":"2024-04-25T13:11:06.564612+0000","last_became_active":"2024-04-25T13:11:05.292474+0000","last_became_peered":"2024-04-25T13:11:05.292474+0000","last_unstale":"2024-04-25T13:11:06.564612+0000","last_undegraded":"2024-04-25T13:11:06.564612+0000","last_fullsized":"2024-04-25T13:11:06.564612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:46.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403943+0000","last_change":"2024-04-25T13:11:04.281104+0000","last_active":"2024-04-25T13:11:06.403943+0000","last_peered":"2024-04-25T13:11:06.403943+0000","last_clean":"2024-04-25T13:11:06.403943+0000","last_became_active":"2024-04-25T13:11:04.280644+0000","last_became_peered":"2024-04-25T13:11:04.280644+0000","last_unstale":"2024-04-25T13:11:06.403943+0000","last_undegraded":"2024-04-25T13:11:06.403943+0000","last_fullsized":"2024-04-25T13:11:06.403943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:15.938218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473117+0000","last_change":"2024-04-25T13:11:05.287532+0000","last_active":"2024-04-25T13:11:06.473117+0000","last_peered":"2024-04-25T13:11:06.473117+0000","last_clean":"2024-04-25T13:11:06.473117+0000","last_became_active":"2024-04-25T13:11:05.287225+0000","last_became_peered":"2024-04-25T13:11:05.287225+0000","last_unstale":"2024-04-25T13:11:06.473117+0000","last_undegraded":"2024-04-25T13:11:06.473117+0000","last_fullsized":"2024-04-25T13:11:06.473117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:23.531317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472773+0000","last_change":"2024-04-25T13:11:04.277567+0000","last_active":"2024-04-25T13:11:06.472773+0000","last_peered":"2024-04-25T13:11:06.472773+0000","last_clean":"2024-04-25T13:11:06.472773+0000","last_became_active":"2024-04-25T13:11:04.277098+0000","last_became_peered":"2024-04-25T13:11:04.277098+0000","last_unstale":"2024-04-25T13:11:06.472773+0000","last_undegraded":"2024-04-25T13:11:06.472773+0000","last_fullsized":"2024-04-25T13:11:06.472773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:23.592989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403672+0000","last_change":"2024-04-25T13:11:05.271586+0000","last_active":"2024-04-25T13:11:06.403672+0000","last_peered":"2024-04-25T13:11:06.403672+0000","last_clean":"2024-04-25T13:11:06.403672+0000","last_became_active":"2024-04-25T13:11:05.271233+0000","last_became_peered":"2024-04-25T13:11:05.271233+0000","last_unstale":"2024-04-25T13:11:06.403672+0000","last_undegraded":"2024-04-25T13:11:06.403672+0000","last_fullsized":"2024-04-25T13:11:06.403672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:53.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.294705+0000","last_change":"2024-04-25T13:11:04.283740+0000","last_active":"2024-04-25T13:11:05.294705+0000","last_peered":"2024-04-25T13:11:05.294705+0000","last_clean":"2024-04-25T13:11:05.294705+0000","last_became_active":"2024-04-25T13:11:04.283428+0000","last_became_peered":"2024-04-25T13:11:04.283428+0000","last_unstale":"2024-04-25T13:11:05.294705+0000","last_undegraded":"2024-04-25T13:11:05.294705+0000","last_fullsized":"2024-04-25T13:11:05.294705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:47.208036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473028+0000","last_change":"2024-04-25T13:11:05.281360+0000","last_active":"2024-04-25T13:11:06.473028+0000","last_peered":"2024-04-25T13:11:06.473028+0000","last_clean":"2024-04-25T13:11:06.473028+0000","last_became_active":"2024-04-25T13:11:05.281048+0000","last_became_peered":"2024-04-25T13:11:05.281048+0000","last_unstale":"2024-04-25T13:11:06.473028+0000","last_undegraded":"2024-04-25T13:11:06.473028+0000","last_fullsized":"2024-04-25T13:11:06.473028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:27.285125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564699+0000","last_change":"2024-04-25T13:11:04.272020+0000","last_active":"2024-04-25T13:11:06.564699+0000","last_peered":"2024-04-25T13:11:06.564699+0000","last_clean":"2024-04-25T13:11:06.564699+0000","last_became_active":"2024-04-25T13:11:04.271472+0000","last_became_peered":"2024-04-25T13:11:04.271472+0000","last_unstale":"2024-04-25T13:11:06.564699+0000","last_undegraded":"2024-04-25T13:11:06.564699+0000","last_fullsized":"2024-04-25T13:11:06.564699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:32.784185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564463+0000","last_change":"2024-04-25T13:11:05.272604+0000","last_active":"2024-04-25T13:11:06.564463+0000","last_peered":"2024-04-25T13:11:06.564463+0000","last_clean":"2024-04-25T13:11:06.564463+0000","last_became_active":"2024-04-25T13:11:05.272246+0000","last_became_peered":"2024-04-25T13:11:05.272246+0000","last_unstale":"2024-04-25T13:11:06.564463+0000","last_undegraded":"2024-04-25T13:11:06.564463+0000","last_fullsized":"2024-04-25T13:11:06.564463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:04.945342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472869+0000","last_change":"2024-04-25T13:11:04.279441+0000","last_active":"2024-04-25T13:11:06.472869+0000","last_peered":"2024-04-25T13:11:06.472869+0000","last_clean":"2024-04-25T13:11:06.472869+0000","last_became_active":"2024-04-25T13:11:04.278940+0000","last_became_peered":"2024-04-25T13:11:04.278940+0000","last_unstale":"2024-04-25T13:11:06.472869+0000","last_undegraded":"2024-04-25T13:11:06.472869+0000","last_fullsized":"2024-04-25T13:11:06.472869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:20.507906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272264+0000","last_change":"2024-04-25T13:11:05.282585+0000","last_active":"2024-04-25T13:11:06.272264+0000","last_peered":"2024-04-25T13:11:06.272264+0000","last_clean":"2024-04-25T13:11:06.272264+0000","last_became_active":"2024-04-25T13:11:05.282248+0000","last_became_peered":"2024-04-25T13:11:05.282248+0000","last_unstale":"2024-04-25T13:11:06.272264+0000","last_undegraded":"2024-04-25T13:11:06.272264+0000","last_fullsized":"2024-04-25T13:11:06.272264+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.671230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.274212+0000","last_change":"2024-04-25T13:11:04.277079+0000","last_active":"2024-04-25T13:11:05.274212+0000","last_peered":"2024-04-25T13:11:05.274212+0000","last_clean":"2024-04-25T13:11:05.274212+0000","last_became_active":"2024-04-25T13:11:04.276694+0000","last_became_peered":"2024-04-25T13:11:04.276694+0000","last_unstale":"2024-04-25T13:11:05.274212+0000","last_undegraded":"2024-04-25T13:11:05.274212+0000","last_fullsized":"2024-04-25T13:11:05.274212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:36.714106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565026+0000","last_change":"2024-04-25T13:11:05.292920+0000","last_active":"2024-04-25T13:11:06.565026+0000","last_peered":"2024-04-25T13:11:06.565026+0000","last_clean":"2024-04-25T13:11:06.565026+0000","last_became_active":"2024-04-25T13:11:05.292556+0000","last_became_peered":"2024-04-25T13:11:05.292556+0000","last_unstale":"2024-04-25T13:11:06.565026+0000","last_undegraded":"2024-04-25T13:11:06.565026+0000","last_fullsized":"2024-04-25T13:11:06.565026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:28.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.267065+0000","last_change":"2024-04-25T13:11:04.277566+0000","last_active":"2024-04-25T13:11:07.267065+0000","last_peered":"2024-04-25T13:11:07.267065+0000","last_clean":"2024-04-25T13:11:07.267065+0000","last_became_active":"2024-04-25T13:11:04.276909+0000","last_became_peered":"2024-04-25T13:11:04.276909+0000","last_unstale":"2024-04-25T13:11:07.267065+0000","last_undegraded":"2024-04-25T13:11:07.267065+0000","last_fullsized":"2024-04-25T13:11:07.267065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:43.907256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565559+0000","last_change":"2024-04-25T13:11:05.273851+0000","last_active":"2024-04-25T13:11:06.565559+0000","last_peered":"2024-04-25T13:11:06.565559+0000","last_clean":"2024-04-25T13:11:06.565559+0000","last_became_active":"2024-04-25T13:11:05.273503+0000","last_became_peered":"2024-04-25T13:11:05.273503+0000","last_unstale":"2024-04-25T13:11:06.565559+0000","last_undegraded":"2024-04-25T13:11:06.565559+0000","last_fullsized":"2024-04-25T13:11:06.565559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:38.532019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473082+0000","last_change":"2024-04-25T13:11:04.279446+0000","last_active":"2024-04-25T13:11:06.473082+0000","last_peered":"2024-04-25T13:11:06.473082+0000","last_clean":"2024-04-25T13:11:06.473082+0000","last_became_active":"2024-04-25T13:11:04.279083+0000","last_became_peered":"2024-04-25T13:11:04.279083+0000","last_unstale":"2024-04-25T13:11:06.473082+0000","last_undegraded":"2024-04-25T13:11:06.473082+0000","last_fullsized":"2024-04-25T13:11:06.473082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:28.719965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565139+0000","last_change":"2024-04-25T13:11:05.283795+0000","last_active":"2024-04-25T13:11:06.565139+0000","last_peered":"2024-04-25T13:11:06.565139+0000","last_clean":"2024-04-25T13:11:06.565139+0000","last_became_active":"2024-04-25T13:11:05.283356+0000","last_became_peered":"2024-04-25T13:11:05.283356+0000","last_unstale":"2024-04-25T13:11:06.565139+0000","last_undegraded":"2024-04-25T13:11:06.565139+0000","last_fullsized":"2024-04-25T13:11:06.565139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:57.095913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'6","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.298202+0000","last_change":"2024-04-25T13:11:04.277100+0000","last_active":"2024-04-25T13:11:05.298202+0000","last_peered":"2024-04-25T13:11:05.298202+0000","last_clean":"2024-04-25T13:11:05.298202+0000","last_became_active":"2024-04-25T13:11:04.276699+0000","last_became_peered":"2024-04-25T13:11:04.276699+0000","last_unstale":"2024-04-25T13:11:05.298202+0000","last_undegraded":"2024-04-25T13:11:05.298202+0000","last_fullsized":"2024-04-25T13:11:05.298202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:54.362024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403726+0000","last_change":"2024-04-25T13:11:05.284175+0000","last_active":"2024-04-25T13:11:06.403726+0000","last_peered":"2024-04-25T13:11:06.403726+0000","last_clean":"2024-04-25T13:11:06.403726+0000","last_became_active":"2024-04-25T13:11:05.283830+0000","last_became_peered":"2024-04-25T13:11:05.283830+0000","last_unstale":"2024-04-25T13:11:06.403726+0000","last_undegraded":"2024-04-25T13:11:06.403726+0000","last_fullsized":"2024-04-25T13:11:06.403726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:18.462117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.291196+0000","last_change":"2024-04-25T13:11:04.277374+0000","last_active":"2024-04-25T13:11:08.291196+0000","last_peered":"2024-04-25T13:11:08.291196+0000","last_clean":"2024-04-25T13:11:08.291196+0000","last_became_active":"2024-04-25T13:11:04.276995+0000","last_became_peered":"2024-04-25T13:11:04.276995+0000","last_unstale":"2024-04-25T13:11:08.291196+0000","last_undegraded":"2024-04-25T13:11:08.291196+0000","last_fullsized":"2024-04-25T13:11:08.291196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:04.856506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.566037+0000","last_change":"2024-04-25T13:11:05.284192+0000","last_active":"2024-04-25T13:11:06.566037+0000","last_peered":"2024-04-25T13:11:06.566037+0000","last_clean":"2024-04-25T13:11:06.566037+0000","last_became_active":"2024-04-25T13:11:05.283877+0000","last_became_peered":"2024-04-25T13:11:05.283877+0000","last_unstale":"2024-04-25T13:11:06.566037+0000","last_undegraded":"2024-04-25T13:11:06.566037+0000","last_fullsized":"2024-04-25T13:11:06.566037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:07.297999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472916+0000","last_change":"2024-04-25T13:11:04.276776+0000","last_active":"2024-04-25T13:11:06.472916+0000","last_peered":"2024-04-25T13:11:06.472916+0000","last_clean":"2024-04-25T13:11:06.472916+0000","last_became_active":"2024-04-25T13:11:04.276302+0000","last_became_peered":"2024-04-25T13:11:04.276302+0000","last_unstale":"2024-04-25T13:11:06.472916+0000","last_undegraded":"2024-04-25T13:11:06.472916+0000","last_fullsized":"2024-04-25T13:11:06.472916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:49.560839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273226+0000","last_change":"2024-04-25T13:11:05.281019+0000","last_active":"2024-04-25T13:11:06.273226+0000","last_peered":"2024-04-25T13:11:06.273226+0000","last_clean":"2024-04-25T13:11:06.273226+0000","last_became_active":"2024-04-25T13:11:05.280566+0000","last_became_peered":"2024-04-25T13:11:05.280566+0000","last_unstale":"2024-04-25T13:11:06.273226+0000","last_undegraded":"2024-04-25T13:11:06.273226+0000","last_fullsized":"2024-04-25T13:11:06.273226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:34.163825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:09.304146+0000","last_change":"2024-04-25T13:11:04.274968+0000","last_active":"2024-04-25T13:11:09.304146+0000","last_peered":"2024-04-25T13:11:09.304146+0000","last_clean":"2024-04-25T13:11:09.304146+0000","last_became_active":"2024-04-25T13:11:04.274617+0000","last_became_peered":"2024-04-25T13:11:04.274617+0000","last_unstale":"2024-04-25T13:11:09.304146+0000","last_undegraded":"2024-04-25T13:11:09.304146+0000","last_fullsized":"2024-04-25T13:11:09.304146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.502451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272999+0000","last_change":"2024-04-25T13:11:05.284103+0000","last_active":"2024-04-25T13:11:06.272999+0000","last_peered":"2024-04-25T13:11:06.272999+0000","last_clean":"2024-04-25T13:11:06.272999+0000","last_became_active":"2024-04-25T13:11:05.283689+0000","last_became_peered":"2024-04-25T13:11:05.283689+0000","last_unstale":"2024-04-25T13:11:06.272999+0000","last_undegraded":"2024-04-25T13:11:06.272999+0000","last_fullsized":"2024-04-25T13:11:06.272999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:09.967889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.295105+0000","last_change":"2024-04-25T13:11:04.281231+0000","last_active":"2024-04-25T13:11:08.295105+0000","last_peered":"2024-04-25T13:11:08.295105+0000","last_clean":"2024-04-25T13:11:08.295105+0000","last_became_active":"2024-04-25T13:11:04.280866+0000","last_became_peered":"2024-04-25T13:11:04.280866+0000","last_unstale":"2024-04-25T13:11:08.295105+0000","last_undegraded":"2024-04-25T13:11:08.295105+0000","last_fullsized":"2024-04-25T13:11:08.295105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:37.284138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472802+0000","last_change":"2024-04-25T13:11:05.283350+0000","last_active":"2024-04-25T13:11:06.472802+0000","last_peered":"2024-04-25T13:11:06.472802+0000","last_clean":"2024-04-25T13:11:06.472802+0000","last_became_active":"2024-04-25T13:11:05.282447+0000","last_became_peered":"2024-04-25T13:11:05.282447+0000","last_unstale":"2024-04-25T13:11:06.472802+0000","last_undegraded":"2024-04-25T13:11:06.472802+0000","last_fullsized":"2024-04-25T13:11:06.472802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:19.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403853+0000","last_change":"2024-04-25T13:11:04.275563+0000","last_active":"2024-04-25T13:11:06.403853+0000","last_peered":"2024-04-25T13:11:06.403853+0000","last_clean":"2024-04-25T13:11:06.403853+0000","last_became_active":"2024-04-25T13:11:04.275151+0000","last_became_peered":"2024-04-25T13:11:04.275151+0000","last_unstale":"2024-04-25T13:11:06.403853+0000","last_undegraded":"2024-04-25T13:11:06.403853+0000","last_fullsized":"2024-04-25T13:11:06.403853+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:27.629485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272933+0000","last_change":"2024-04-25T13:11:05.280723+0000","last_active":"2024-04-25T13:11:06.272933+0000","last_peered":"2024-04-25T13:11:06.272933+0000","last_clean":"2024-04-25T13:11:06.272933+0000","last_became_active":"2024-04-25T13:11:05.280271+0000","last_became_peered":"2024-04-25T13:11:05.280271+0000","last_unstale":"2024-04-25T13:11:06.272933+0000","last_undegraded":"2024-04-25T13:11:06.272933+0000","last_fullsized":"2024-04-25T13:11:06.272933+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:18.099610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272150+0000","last_change":"2024-04-25T13:11:04.275774+0000","last_active":"2024-04-25T13:11:06.272150+0000","last_peered":"2024-04-25T13:11:06.272150+0000","last_clean":"2024-04-25T13:11:06.272150+0000","last_became_active":"2024-04-25T13:11:04.275459+0000","last_became_peered":"2024-04-25T13:11:04.275459+0000","last_unstale":"2024-04-25T13:11:06.272150+0000","last_undegraded":"2024-04-25T13:11:06.272150+0000","last_fullsized":"2024-04-25T13:11:06.272150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.465906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.285696+0000","last_change":"2024-04-25T13:11:05.285237+0000","last_active":"2024-04-25T13:11:05.285696+0000","last_peered":"2024-04-25T13:11:05.285696+0000","last_clean":"2024-04-25T13:11:05.285696+0000","last_became_active":"2024-04-25T13:11:05.284875+0000","last_became_peered":"2024-04-25T13:11:05.284875+0000","last_unstale":"2024-04-25T13:11:05.285696+0000","last_undegraded":"2024-04-25T13:11:05.285696+0000","last_fullsized":"2024-04-25T13:11:05.285696+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:12.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272870+0000","last_change":"2024-04-25T13:11:04.273216+0000","last_active":"2024-04-25T13:11:06.272870+0000","last_peered":"2024-04-25T13:11:06.272870+0000","last_clean":"2024-04-25T13:11:06.272870+0000","last_became_active":"2024-04-25T13:11:04.272850+0000","last_became_peered":"2024-04-25T13:11:04.272850+0000","last_unstale":"2024-04-25T13:11:06.272870+0000","last_undegraded":"2024-04-25T13:11:06.272870+0000","last_fullsized":"2024-04-25T13:11:06.272870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:53.566971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.290761+0000","last_change":"2024-04-25T13:11:05.289951+0000","last_active":"2024-04-25T13:11:05.290761+0000","last_peered":"2024-04-25T13:11:05.290761+0000","last_clean":"2024-04-25T13:11:05.290761+0000","last_became_active":"2024-04-25T13:11:05.289575+0000","last_became_peered":"2024-04-25T13:11:05.289575+0000","last_unstale":"2024-04-25T13:11:05.290761+0000","last_undegraded":"2024-04-25T13:11:05.290761+0000","last_fullsized":"2024-04-25T13:11:05.290761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:33.843458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473196+0000","last_change":"2024-04-25T13:11:05.278811+0000","last_active":"2024-04-25T13:11:06.473196+0000","last_peered":"2024-04-25T13:11:06.473196+0000","last_clean":"2024-04-25T13:11:06.473196+0000","last_became_active":"2024-04-25T13:11:05.278476+0000","last_became_peered":"2024-04-25T13:11:05.278476+0000","last_unstale":"2024-04-25T13:11:06.473196+0000","last_undegraded":"2024-04-25T13:11:06.473196+0000","last_fullsized":"2024-04-25T13:11:06.473196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:40.656046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564652+0000","last_change":"2024-04-25T13:11:04.277793+0000","last_active":"2024-04-25T13:11:06.564652+0000","last_peered":"2024-04-25T13:11:06.564652+0000","last_clean":"2024-04-25T13:11:06.564652+0000","last_became_active":"2024-04-25T13:11:04.277254+0000","last_became_peered":"2024-04-25T13:11:04.277254+0000","last_unstale":"2024-04-25T13:11:06.564652+0000","last_undegraded":"2024-04-25T13:11:06.564652+0000","last_fullsized":"2024-04-25T13:11:06.564652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:19.481074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564832+0000","last_change":"2024-04-25T13:11:05.285358+0000","last_active":"2024-04-25T13:11:06.564832+0000","last_peered":"2024-04-25T13:11:06.564832+0000","last_clean":"2024-04-25T13:11:06.564832+0000","last_became_active":"2024-04-25T13:11:05.284641+0000","last_became_peered":"2024-04-25T13:11:05.284641+0000","last_unstale":"2024-04-25T13:11:06.564832+0000","last_undegraded":"2024-04-25T13:11:06.564832+0000","last_fullsized":"2024-04-25T13:11:06.564832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:33.245556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:09.300025+0000","last_change":"2024-04-25T13:11:04.272294+0000","last_active":"2024-04-25T13:11:09.300025+0000","last_peered":"2024-04-25T13:11:09.300025+0000","last_clean":"2024-04-25T13:11:09.300025+0000","last_became_active":"2024-04-25T13:11:04.271898+0000","last_became_peered":"2024-04-25T13:11:04.271898+0000","last_unstale":"2024-04-25T13:11:09.300025+0000","last_undegraded":"2024-04-25T13:11:09.300025+0000","last_fullsized":"2024-04-25T13:11:09.300025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:29.130773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273010+0000","last_change":"2024-04-25T13:09:54.520980+0000","last_active":"2024-04-25T13:11:06.273010+0000","last_peered":"2024-04-25T13:11:06.273010+0000","last_clean":"2024-04-25T13:11:06.273010+0000","last_became_active":"2024-04-25T13:09:54.520446+0000","last_became_peered":"2024-04-25T13:09:54.520446+0000","last_unstale":"2024-04-25T13:11:06.273010+0000","last_undegraded":"2024-04-25T13:11:06.273010+0000","last_fullsized":"2024-04-25T13:11:06.273010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:06:33.347730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:06:33.347730+0000","last_clean_scrub_stamp":"2024-04-25T13:06:33.347730+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:00.996641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564588+0000","last_change":"2024-04-25T13:11:05.279449+0000","last_active":"2024-04-25T13:11:06.564588+0000","last_peered":"2024-04-25T13:11:06.564588+0000","last_clean":"2024-04-25T13:11:06.564588+0000","last_became_active":"2024-04-25T13:11:05.278982+0000","last_became_peered":"2024-04-25T13:11:05.278982+0000","last_unstale":"2024-04-25T13:11:06.564588+0000","last_undegraded":"2024-04-25T13:11:06.564588+0000","last_fullsized":"2024-04-25T13:11:06.564588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:44.647967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.293696+0000","last_change":"2024-04-25T13:11:04.279131+0000","last_active":"2024-04-25T13:11:08.293696+0000","last_peered":"2024-04-25T13:11:08.293696+0000","last_clean":"2024-04-25T13:11:08.293696+0000","last_became_active":"2024-04-25T13:11:04.278801+0000","last_became_peered":"2024-04-25T13:11:04.278801+0000","last_unstale":"2024-04-25T13:11:08.293696+0000","last_undegraded":"2024-04-25T13:11:08.293696+0000","last_fullsized":"2024-04-25T13:11:08.293696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:06.341497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.265750+0000","last_change":"2024-04-25T13:11:04.275784+0000","last_active":"2024-04-25T13:11:07.265750+0000","last_peered":"2024-04-25T13:11:07.265750+0000","last_clean":"2024-04-25T13:11:07.265750+0000","last_became_active":"2024-04-25T13:11:04.275487+0000","last_became_peered":"2024-04-25T13:11:04.275487+0000","last_unstale":"2024-04-25T13:11:07.265750+0000","last_undegraded":"2024-04-25T13:11:07.265750+0000","last_fullsized":"2024-04-25T13:11:07.265750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.405056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565627+0000","last_change":"2024-04-25T13:11:05.275854+0000","last_active":"2024-04-25T13:11:06.565627+0000","last_peered":"2024-04-25T13:11:06.565627+0000","last_clean":"2024-04-25T13:11:06.565627+0000","last_became_active":"2024-04-25T13:11:05.275527+0000","last_became_peered":"2024-04-25T13:11:05.275527+0000","last_unstale":"2024-04-25T13:11:06.565627+0000","last_undegraded":"2024-04-25T13:11:06.565627+0000","last_fullsized":"2024-04-25T13:11:06.565627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:42.300458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.294489+0000","last_change":"2024-04-25T13:11:04.279415+0000","last_active":"2024-04-25T13:11:08.294489+0000","last_peered":"2024-04-25T13:11:08.294489+0000","last_clean":"2024-04-25T13:11:08.294489+0000","last_became_active":"2024-04-25T13:11:04.279119+0000","last_became_peered":"2024-04-25T13:11:04.279119+0000","last_unstale":"2024-04-25T13:11:08.294489+0000","last_undegraded":"2024-04-25T13:11:08.294489+0000","last_fullsized":"2024-04-25T13:11:08.294489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:18.917509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565327+0000","last_change":"2024-04-25T13:11:05.276317+0000","last_active":"2024-04-25T13:11:06.565327+0000","last_peered":"2024-04-25T13:11:06.565327+0000","last_clean":"2024-04-25T13:11:06.565327+0000","last_became_active":"2024-04-25T13:11:05.275781+0000","last_became_peered":"2024-04-25T13:11:05.275781+0000","last_unstale":"2024-04-25T13:11:06.565327+0000","last_undegraded":"2024-04-25T13:11:06.565327+0000","last_fullsized":"2024-04-25T13:11:06.565327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:41.628388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.265929+0000","last_change":"2024-04-25T13:11:04.273617+0000","last_active":"2024-04-25T13:11:07.265929+0000","last_peered":"2024-04-25T13:11:07.265929+0000","last_clean":"2024-04-25T13:11:07.265929+0000","last_became_active":"2024-04-25T13:11:04.272749+0000","last_became_peered":"2024-04-25T13:11:04.272749+0000","last_unstale":"2024-04-25T13:11:07.265929+0000","last_undegraded":"2024-04-25T13:11:07.265929+0000","last_fullsized":"2024-04-25T13:11:07.265929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:14.484215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565450+0000","last_change":"2024-04-25T13:11:05.274423+0000","last_active":"2024-04-25T13:11:06.565450+0000","last_peered":"2024-04-25T13:11:06.565450+0000","last_clean":"2024-04-25T13:11:06.565450+0000","last_became_active":"2024-04-25T13:11:05.273180+0000","last_became_peered":"2024-04-25T13:11:05.273180+0000","last_unstale":"2024-04-25T13:11:06.565450+0000","last_undegraded":"2024-04-25T13:11:06.565450+0000","last_fullsized":"2024-04-25T13:11:06.565450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:51.082000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.266877+0000","last_change":"2024-04-25T13:11:04.275671+0000","last_active":"2024-04-25T13:11:07.266877+0000","last_peered":"2024-04-25T13:11:07.266877+0000","last_clean":"2024-04-25T13:11:07.266877+0000","last_became_active":"2024-04-25T13:11:04.275357+0000","last_became_peered":"2024-04-25T13:11:04.275357+0000","last_unstale":"2024-04-25T13:11:07.266877+0000","last_undegraded":"2024-04-25T13:11:07.266877+0000","last_fullsized":"2024-04-25T13:11:07.266877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:55.552043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403484+0000","last_change":"2024-04-25T13:11:05.282327+0000","last_active":"2024-04-25T13:11:06.403484+0000","last_peered":"2024-04-25T13:11:06.403484+0000","last_clean":"2024-04-25T13:11:06.403484+0000","last_became_active":"2024-04-25T13:11:05.281903+0000","last_became_peered":"2024-04-25T13:11:05.281903+0000","last_unstale":"2024-04-25T13:11:06.403484+0000","last_undegraded":"2024-04-25T13:11:06.403484+0000","last_fullsized":"2024-04-25T13:11:06.403484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:40.490726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564738+0000","last_change":"2024-04-25T13:11:04.273065+0000","last_active":"2024-04-25T13:11:06.564738+0000","last_peered":"2024-04-25T13:11:06.564738+0000","last_clean":"2024-04-25T13:11:06.564738+0000","last_became_active":"2024-04-25T13:11:04.272723+0000","last_became_peered":"2024-04-25T13:11:04.272723+0000","last_unstale":"2024-04-25T13:11:06.564738+0000","last_undegraded":"2024-04-25T13:11:06.564738+0000","last_fullsized":"2024-04-25T13:11:06.564738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:56.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272593+0000","last_change":"2024-04-25T13:11:05.276845+0000","last_active":"2024-04-25T13:11:06.272593+0000","last_peered":"2024-04-25T13:11:06.272593+0000","last_clean":"2024-04-25T13:11:06.272593+0000","last_became_active":"2024-04-25T13:11:05.276466+0000","last_became_peered":"2024-04-25T13:11:05.276466+0000","last_unstale":"2024-04-25T13:11:06.272593+0000","last_undegraded":"2024-04-25T13:11:06.272593+0000","last_fullsized":"2024-04-25T13:11:06.272593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:55.934335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564489+0000","last_change":"2024-04-25T13:11:04.278970+0000","last_active":"2024-04-25T13:11:06.564489+0000","last_peered":"2024-04-25T13:11:06.564489+0000","last_clean":"2024-04-25T13:11:06.564489+0000","last_became_active":"2024-04-25T13:11:04.278594+0000","last_became_peered":"2024-04-25T13:11:04.278594+0000","last_unstale":"2024-04-25T13:11:06.564489+0000","last_undegraded":"2024-04-25T13:11:06.564489+0000","last_fullsized":"2024-04-25T13:11:06.564489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:31.099566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565810+0000","last_change":"2024-04-25T13:11:05.281008+0000","last_active":"2024-04-25T13:11:06.565810+0000","last_peered":"2024-04-25T13:11:06.565810+0000","last_clean":"2024-04-25T13:11:06.565810+0000","last_became_active":"2024-04-25T13:11:05.280581+0000","last_became_peered":"2024-04-25T13:11:05.280581+0000","last_unstale":"2024-04-25T13:11:06.565810+0000","last_undegraded":"2024-04-25T13:11:06.565810+0000","last_fullsized":"2024-04-25T13:11:06.565810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:28.636608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.267053+0000","last_change":"2024-04-25T13:11:04.281969+0000","last_active":"2024-04-25T13:11:07.267053+0000","last_peered":"2024-04-25T13:11:07.267053+0000","last_clean":"2024-04-25T13:11:07.267053+0000","last_became_active":"2024-04-25T13:11:04.281501+0000","last_became_peered":"2024-04-25T13:11:04.281501+0000","last_unstale":"2024-04-25T13:11:07.267053+0000","last_undegraded":"2024-04-25T13:11:07.267053+0000","last_fullsized":"2024-04-25T13:11:07.267053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:36.494799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273356+0000","last_change":"2024-04-25T13:11:05.288121+0000","last_active":"2024-04-25T13:11:06.273356+0000","last_peered":"2024-04-25T13:11:06.273356+0000","last_clean":"2024-04-25T13:11:06.273356+0000","last_became_active":"2024-04-25T13:11:05.287816+0000","last_became_peered":"2024-04-25T13:11:05.287816+0000","last_unstale":"2024-04-25T13:11:06.273356+0000","last_undegraded":"2024-04-25T13:11:06.273356+0000","last_fullsized":"2024-04-25T13:11:06.273356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:02.855646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.294750+0000","last_change":"2024-04-25T13:11:04.279883+0000","last_active":"2024-04-25T13:11:08.294750+0000","last_peered":"2024-04-25T13:11:08.294750+0000","last_clean":"2024-04-25T13:11:08.294750+0000","last_became_active":"2024-04-25T13:11:04.279408+0000","last_became_peered":"2024-04-25T13:11:04.279408+0000","last_unstale":"2024-04-25T13:11:08.294750+0000","last_undegraded":"2024-04-25T13:11:08.294750+0000","last_fullsized":"2024-04-25T13:11:08.294750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:42.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273320+0000","last_change":"2024-04-25T13:11:05.284845+0000","last_active":"2024-04-25T13:11:06.273320+0000","last_peered":"2024-04-25T13:11:06.273320+0000","last_clean":"2024-04-25T13:11:06.273320+0000","last_became_active":"2024-04-25T13:11:05.284300+0000","last_became_peered":"2024-04-25T13:11:05.284300+0000","last_unstale":"2024-04-25T13:11:06.273320+0000","last_undegraded":"2024-04-25T13:11:06.273320+0000","last_fullsized":"2024-04-25T13:11:06.273320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:54.233049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472954+0000","last_change":"2024-04-25T13:11:04.274075+0000","last_active":"2024-04-25T13:11:06.472954+0000","last_peered":"2024-04-25T13:11:06.472954+0000","last_clean":"2024-04-25T13:11:06.472954+0000","last_became_active":"2024-04-25T13:11:04.273624+0000","last_became_peered":"2024-04-25T13:11:04.273624+0000","last_unstale":"2024-04-25T13:11:06.472954+0000","last_undegraded":"2024-04-25T13:11:06.472954+0000","last_fullsized":"2024-04-25T13:11:06.472954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:52.661939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273014+0000","last_change":"2024-04-25T13:11:05.283181+0000","last_active":"2024-04-25T13:11:06.273014+0000","last_peered":"2024-04-25T13:11:06.273014+0000","last_clean":"2024-04-25T13:11:06.273014+0000","last_became_active":"2024-04-25T13:11:05.282848+0000","last_became_peered":"2024-04-25T13:11:05.282848+0000","last_unstale":"2024-04-25T13:11:06.273014+0000","last_undegraded":"2024-04-25T13:11:06.273014+0000","last_fullsized":"2024-04-25T13:11:06.273014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:21.372069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565654+0000","last_change":"2024-04-25T13:11:04.270691+0000","last_active":"2024-04-25T13:11:06.565654+0000","last_peered":"2024-04-25T13:11:06.565654+0000","last_clean":"2024-04-25T13:11:06.565654+0000","last_became_active":"2024-04-25T13:11:04.270337+0000","last_became_peered":"2024-04-25T13:11:04.270337+0000","last_unstale":"2024-04-25T13:11:06.565654+0000","last_undegraded":"2024-04-25T13:11:06.565654+0000","last_fullsized":"2024-04-25T13:11:06.565654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:05.632472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273400+0000","last_change":"2024-04-25T13:11:05.284284+0000","last_active":"2024-04-25T13:11:06.273400+0000","last_peered":"2024-04-25T13:11:06.273400+0000","last_clean":"2024-04-25T13:11:06.273400+0000","last_became_active":"2024-04-25T13:11:05.283952+0000","last_became_peered":"2024-04-25T13:11:05.283952+0000","last_unstale":"2024-04-25T13:11:06.273400+0000","last_undegraded":"2024-04-25T13:11:06.273400+0000","last_fullsized":"2024-04-25T13:11:06.273400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:32.396183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272662+0000","last_change":"2024-04-25T13:11:05.278064+0000","last_active":"2024-04-25T13:11:06.272662+0000","last_peered":"2024-04-25T13:11:06.272662+0000","last_clean":"2024-04-25T13:11:06.272662+0000","last_became_active":"2024-04-25T13:11:05.277680+0000","last_became_peered":"2024-04-25T13:11:05.277680+0000","last_unstale":"2024-04-25T13:11:06.272662+0000","last_undegraded":"2024-04-25T13:11:06.272662+0000","last_fullsized":"2024-04-25T13:11:06.272662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:32.048667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564662+0000","last_change":"2024-04-25T13:11:05.278432+0000","last_active":"2024-04-25T13:11:06.564662+0000","last_peered":"2024-04-25T13:11:06.564662+0000","last_clean":"2024-04-25T13:11:06.564662+0000","last_became_active":"2024-04-25T13:11:05.278089+0000","last_became_peered":"2024-04-25T13:11:05.278089+0000","last_unstale":"2024-04-25T13:11:06.564662+0000","last_undegraded":"2024-04-25T13:11:06.564662+0000","last_fullsized":"2024-04-25T13:11:06.564662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:49.505681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564676+0000","last_change":"2024-04-25T13:11:05.278072+0000","last_active":"2024-04-25T13:11:06.564676+0000","last_peered":"2024-04-25T13:11:06.564676+0000","last_clean":"2024-04-25T13:11:06.564676+0000","last_became_active":"2024-04-25T13:11:05.277720+0000","last_became_peered":"2024-04-25T13:11:05.277720+0000","last_unstale":"2024-04-25T13:11:06.564676+0000","last_undegraded":"2024-04-25T13:11:06.564676+0000","last_fullsized":"2024-04-25T13:11:06.564676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:14.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564955+0000","last_change":"2024-04-25T13:11:05.272611+0000","last_active":"2024-04-25T13:11:06.564955+0000","last_peered":"2024-04-25T13:11:06.564955+0000","last_clean":"2024-04-25T13:11:06.564955+0000","last_became_active":"2024-04-25T13:11:05.272318+0000","last_became_peered":"2024-04-25T13:11:05.272318+0000","last_unstale":"2024-04-25T13:11:06.564955+0000","last_undegraded":"2024-04-25T13:11:06.564955+0000","last_fullsized":"2024-04-25T13:11:06.564955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:52.796827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565769+0000","last_change":"2024-04-25T13:11:05.281146+0000","last_active":"2024-04-25T13:11:06.565769+0000","last_peered":"2024-04-25T13:11:06.565769+0000","last_clean":"2024-04-25T13:11:06.565769+0000","last_became_active":"2024-04-25T13:11:05.280897+0000","last_became_peered":"2024-04-25T13:11:05.280897+0000","last_unstale":"2024-04-25T13:11:06.565769+0000","last_undegraded":"2024-04-25T13:11:06.565769+0000","last_fullsized":"2024-04-25T13:11:06.565769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:27.348411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272877+0000","last_change":"2024-04-25T13:11:05.275482+0000","last_active":"2024-04-25T13:11:06.272877+0000","last_peered":"2024-04-25T13:11:06.272877+0000","last_clean":"2024-04-25T13:11:06.272877+0000","last_became_active":"2024-04-25T13:11:05.275066+0000","last_became_peered":"2024-04-25T13:11:05.275066+0000","last_unstale":"2024-04-25T13:11:06.272877+0000","last_undegraded":"2024-04-25T13:11:06.272877+0000","last_fullsized":"2024-04-25T13:11:06.272877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:27.286105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565058+0000","last_change":"2024-04-25T13:11:05.287931+0000","last_active":"2024-04-25T13:11:06.565058+0000","last_peered":"2024-04-25T13:11:06.565058+0000","last_clean":"2024-04-25T13:11:06.565058+0000","last_became_active":"2024-04-25T13:11:05.287540+0000","last_became_peered":"2024-04-25T13:11:05.287540+0000","last_unstale":"2024-04-25T13:11:06.565058+0000","last_undegraded":"2024-04-25T13:11:06.565058+0000","last_fullsized":"2024-04-25T13:11:06.565058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:16.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565105+0000","last_change":"2024-04-25T13:11:05.274197+0000","last_active":"2024-04-25T13:11:06.565105+0000","last_peered":"2024-04-25T13:11:06.565105+0000","last_clean":"2024-04-25T13:11:06.565105+0000","last_became_active":"2024-04-25T13:11:05.273236+0000","last_became_peered":"2024-04-25T13:11:05.273236+0000","last_unstale":"2024-04-25T13:11:06.565105+0000","last_undegraded":"2024-04-25T13:11:06.565105+0000","last_fullsized":"2024-04-25T13:11:06.565105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:55.471002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473103+0000","last_change":"2024-04-25T13:11:05.282693+0000","last_active":"2024-04-25T13:11:06.473103+0000","last_peered":"2024-04-25T13:11:06.473103+0000","last_clean":"2024-04-25T13:11:06.473103+0000","last_became_active":"2024-04-25T13:11:05.280002+0000","last_became_peered":"2024-04-25T13:11:05.280002+0000","last_unstale":"2024-04-25T13:11:06.473103+0000","last_undegraded":"2024-04-25T13:11:06.473103+0000","last_fullsized":"2024-04-25T13:11:06.473103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:08.661270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564671+0000","last_change":"2024-04-25T13:11:05.274581+0000","last_active":"2024-04-25T13:11:06.564671+0000","last_peered":"2024-04-25T13:11:06.564671+0000","last_clean":"2024-04-25T13:11:06.564671+0000","last_became_active":"2024-04-25T13:11:05.273634+0000","last_became_peered":"2024-04-25T13:11:05.273634+0000","last_unstale":"2024-04-25T13:11:06.564671+0000","last_undegraded":"2024-04-25T13:11:06.564671+0000","last_fullsized":"2024-04-25T13:11:06.564671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:13.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564786+0000","last_change":"2024-04-25T13:11:05.284581+0000","last_active":"2024-04-25T13:11:06.564786+0000","last_peered":"2024-04-25T13:11:06.564786+0000","last_clean":"2024-04-25T13:11:06.564786+0000","last_became_active":"2024-04-25T13:11:05.283032+0000","last_became_peered":"2024-04-25T13:11:05.283032+0000","last_unstale":"2024-04-25T13:11:06.564786+0000","last_undegraded":"2024-04-25T13:11:06.564786+0000","last_fullsized":"2024-04-25T13:11:06.564786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:12.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.566245+0000","last_change":"2024-04-25T13:11:05.285081+0000","last_active":"2024-04-25T13:11:06.566245+0000","last_peered":"2024-04-25T13:11:06.566245+0000","last_clean":"2024-04-25T13:11:06.566245+0000","last_became_active":"2024-04-25T13:11:05.283717+0000","last_became_peered":"2024-04-25T13:11:05.283717+0000","last_unstale":"2024-04-25T13:11:06.566245+0000","last_undegraded":"2024-04-25T13:11:06.566245+0000","last_fullsized":"2024-04-25T13:11:06.566245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:46.381684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272668+0000","last_change":"2024-04-25T13:11:05.278362+0000","last_active":"2024-04-25T13:11:06.272668+0000","last_peered":"2024-04-25T13:11:06.272668+0000","last_clean":"2024-04-25T13:11:06.272668+0000","last_became_active":"2024-04-25T13:11:05.277985+0000","last_became_peered":"2024-04-25T13:11:05.277985+0000","last_unstale":"2024-04-25T13:11:06.272668+0000","last_undegraded":"2024-04-25T13:11:06.272668+0000","last_fullsized":"2024-04-25T13:11:06.272668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:45.142805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3748,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":98,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":19709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":456042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1556480,"data_stored":1039064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743447,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":456278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":456198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1047620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":446890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":463672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":456938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":457728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":455564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920816,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36848,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708304,"statfs":{"total":95995035648,"available":95957303296,"internally_reserved":0,"allocated":1544192,"data_stored":1037440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:11:11.997 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:11:12.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.130+0000 7fd6aede2640 1 -- 172.21.15.72:0/1954319522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 msgr2=0x7fd6a8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.130+0000 7fd6aede2640 1 --2- 172.21.15.72:0/1954319522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 0x7fd6a8102460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd694009900 tx=0x7fd69402a410 comp rx=0 tx=0).stop 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.130+0000 7fd6aede2640 1 -- 172.21.15.72:0/1954319522 shutdown_connections 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.130+0000 7fd6aede2640 1 --2- 172.21.15.72:0/1954319522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a81029a0 0x7fd6a81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.130+0000 7fd6aede2640 1 --2- 172.21.15.72:0/1954319522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 0x7fd6a8102460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.130+0000 7fd6aede2640 1 --2- 172.21.15.72:0/1954319522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a8100e90 0x7fd6a81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.130+0000 7fd6aede2640 1 -- 172.21.15.72:0/1954319522 >> 172.21.15.72:0/1954319522 conn(0x7fd6a80fc400 msgr2=0x7fd6a80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.130+0000 7fd6aede2640 1 -- 172.21.15.72:0/1954319522 shutdown_connections 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.130+0000 7fd6aede2640 1 -- 172.21.15.72:0/1954319522 wait complete. 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6aede2640 1 Processor -- start 2024-04-25T13:11:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6aede2640 1 -- start start 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6aede2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8100e90 0x7fd6a81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6aede2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 0x7fd6a8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6aede2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a81029a0 0x7fd6a819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6aede2640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a810ca40 con 0x7fd6a8101fe0 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6aede2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a810c8c0 con 0x7fd6a8100e90 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6aede2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a810cbc0 con 0x7fd6a81029a0 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6a7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 0x7fd6a8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6a7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 0x7fd6a8198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56454/0 (socket says 172.21.15.72:56454) 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6a7fff640 1 -- 172.21.15.72:0/3776937329 learned_addr learned my addr 172.21.15.72:0/3776937329 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6ad37f640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a81029a0 0x7fd6a819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6acb7e640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8100e90 0x7fd6a81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6a7fff640 1 -- 172.21.15.72:0/3776937329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a81029a0 msgr2=0x7fd6a819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6a7fff640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a81029a0 0x7fd6a819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6a7fff640 1 -- 172.21.15.72:0/3776937329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8100e90 msgr2=0x7fd6a81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6a7fff640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8100e90 0x7fd6a81982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.131+0000 7fd6a7fff640 1 -- 172.21.15.72:0/3776937329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a819fee0 con 0x7fd6a8101fe0 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.132+0000 7fd6ad37f640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a81029a0 0x7fd6a819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.132+0000 7fd6acb7e640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8100e90 0x7fd6a81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.132+0000 7fd6a7fff640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 0x7fd6a8198800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd694006010 tx=0x7fd694004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.132+0000 7fd6a5ffb640 1 -- 172.21.15.72:0/3776937329 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69403f030 con 0x7fd6a8101fe0 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.132+0000 7fd6a5ffb640 1 -- 172.21.15.72:0/3776937329 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd694003cb0 con 0x7fd6a8101fe0 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.132+0000 7fd6a5ffb640 1 -- 172.21.15.72:0/3776937329 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694035070 con 0x7fd6a8101fe0 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.132+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a81a0170 con 0x7fd6a8101fe0 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.132+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a81a06e0 con 0x7fd6a8101fe0 2024-04-25T13:11:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.133+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a81a0940 con 0x7fd6a8101fe0 2024-04-25T13:11:12.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.134+0000 7fd6a5ffb640 1 -- 172.21.15.72:0/3776937329 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd694042030 con 0x7fd6a8101fe0 2024-04-25T13:11:12.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.135+0000 7fd6a5ffb640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd6980a3c40 0x7fd6980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:12.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.135+0000 7fd6acb7e640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd6980a3c40 0x7fd6980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.135+0000 7fd6a5ffb640 1 -- 172.21.15.72:0/3776937329 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd6940f0ba0 con 0x7fd6a8101fe0 2024-04-25T13:11:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.135+0000 7fd6acb7e640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd6980a3c40 0x7fd6980a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd69c006fd0 tx=0x7fd69c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:12.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.139+0000 7fd6a5ffb640 1 -- 172.21.15.72:0/3776937329 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd694003e60 con 0x7fd6a8101fe0 2024-04-25T13:11:12.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.320+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6a8069860 con 0x7fd6980a3c40 2024-04-25T13:11:12.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.327+0000 7fd6a5ffb640 1 -- 172.21.15.72:0/3776937329 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258751 (secure 0 0 0) 0x7fd6a8069860 con 0x7fd6980a3c40 2024-04-25T13:11:12.329 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:12.331 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-25T13:11:12.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd6980a3c40 msgr2=0x7fd6980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:12.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd6980a3c40 0x7fd6980a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd69c006fd0 tx=0x7fd69c008040 comp rx=0 tx=0).stop 2024-04-25T13:11:12.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 msgr2=0x7fd6a8198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:12.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 0x7fd6a8198800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd694006010 tx=0x7fd694004430 comp rx=0 tx=0).stop 2024-04-25T13:11:12.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 shutdown_connections 2024-04-25T13:11:12.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd6980a3c40 0x7fd6980a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a81029a0 0x7fd6a819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6a8101fe0 0x7fd6a8198800 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 --2- 172.21.15.72:0/3776937329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8100e90 0x7fd6a81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 >> 172.21.15.72:0/3776937329 conn(0x7fd6a80fc400 msgr2=0x7fd6a8069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:12.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.330+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 shutdown_connections 2024-04-25T13:11:12.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:12.331+0000 7fd6aede2640 1 -- 172.21.15.72:0/3776937329 wait complete. 2024-04-25T13:11:12.341 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-25T13:11:11.125635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594116,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":150,"num_read_kb":131,"num_write":279,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383012,"kb_used_data":13024,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558812,"statfs":{"total":1151940427776,"available":1151548223488,"internally_reserved":0,"allocated":13336576,"data_stored":7229538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3748,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":98,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.869077"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273029+0000","last_change":"2024-04-25T13:11:05.285490+0000","last_active":"2024-04-25T13:11:06.273029+0000","last_peered":"2024-04-25T13:11:06.273029+0000","last_clean":"2024-04-25T13:11:06.273029+0000","last_became_active":"2024-04-25T13:11:05.285066+0000","last_became_peered":"2024-04-25T13:11:05.285066+0000","last_unstale":"2024-04-25T13:11:06.273029+0000","last_undegraded":"2024-04-25T13:11:06.273029+0000","last_fullsized":"2024-04-25T13:11:06.273029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:30.002823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565459+0000","last_change":"2024-04-25T13:11:05.292670+0000","last_active":"2024-04-25T13:11:06.565459+0000","last_peered":"2024-04-25T13:11:06.565459+0000","last_clean":"2024-04-25T13:11:06.565459+0000","last_became_active":"2024-04-25T13:11:05.292279+0000","last_became_peered":"2024-04-25T13:11:05.292279+0000","last_unstale":"2024-04-25T13:11:06.565459+0000","last_undegraded":"2024-04-25T13:11:06.565459+0000","last_fullsized":"2024-04-25T13:11:06.565459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:26.439874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565225+0000","last_change":"2024-04-25T13:11:05.284228+0000","last_active":"2024-04-25T13:11:06.565225+0000","last_peered":"2024-04-25T13:11:06.565225+0000","last_clean":"2024-04-25T13:11:06.565225+0000","last_became_active":"2024-04-25T13:11:05.283877+0000","last_became_peered":"2024-04-25T13:11:05.283877+0000","last_unstale":"2024-04-25T13:11:06.565225+0000","last_undegraded":"2024-04-25T13:11:06.565225+0000","last_fullsized":"2024-04-25T13:11:06.565225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:19.768393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565866+0000","last_change":"2024-04-25T13:11:05.276516+0000","last_active":"2024-04-25T13:11:06.565866+0000","last_peered":"2024-04-25T13:11:06.565866+0000","last_clean":"2024-04-25T13:11:06.565866+0000","last_became_active":"2024-04-25T13:11:05.276174+0000","last_became_peered":"2024-04-25T13:11:05.276174+0000","last_unstale":"2024-04-25T13:11:06.565866+0000","last_undegraded":"2024-04-25T13:11:06.565866+0000","last_fullsized":"2024-04-25T13:11:06.565866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:27.801716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403554+0000","last_change":"2024-04-25T13:11:05.279010+0000","last_active":"2024-04-25T13:11:06.403554+0000","last_peered":"2024-04-25T13:11:06.403554+0000","last_clean":"2024-04-25T13:11:06.403554+0000","last_became_active":"2024-04-25T13:11:05.276879+0000","last_became_peered":"2024-04-25T13:11:05.276879+0000","last_unstale":"2024-04-25T13:11:06.403554+0000","last_undegraded":"2024-04-25T13:11:06.403554+0000","last_fullsized":"2024-04-25T13:11:06.403554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:14.191090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.292741+0000","last_change":"2024-04-25T13:11:05.291530+0000","last_active":"2024-04-25T13:11:05.292741+0000","last_peered":"2024-04-25T13:11:05.292741+0000","last_clean":"2024-04-25T13:11:05.292741+0000","last_became_active":"2024-04-25T13:11:05.291119+0000","last_became_peered":"2024-04-25T13:11:05.291119+0000","last_unstale":"2024-04-25T13:11:05.292741+0000","last_undegraded":"2024-04-25T13:11:05.292741+0000","last_fullsized":"2024-04-25T13:11:05.292741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:08.498624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272843+0000","last_change":"2024-04-25T13:11:05.281816+0000","last_active":"2024-04-25T13:11:06.272843+0000","last_peered":"2024-04-25T13:11:06.272843+0000","last_clean":"2024-04-25T13:11:06.272843+0000","last_became_active":"2024-04-25T13:11:05.281504+0000","last_became_peered":"2024-04-25T13:11:05.281504+0000","last_unstale":"2024-04-25T13:11:06.272843+0000","last_undegraded":"2024-04-25T13:11:06.272843+0000","last_fullsized":"2024-04-25T13:11:06.272843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.724238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564627+0000","last_change":"2024-04-25T13:11:05.279320+0000","last_active":"2024-04-25T13:11:06.564627+0000","last_peered":"2024-04-25T13:11:06.564627+0000","last_clean":"2024-04-25T13:11:06.564627+0000","last_became_active":"2024-04-25T13:11:05.278995+0000","last_became_peered":"2024-04-25T13:11:05.278995+0000","last_unstale":"2024-04-25T13:11:06.564627+0000","last_undegraded":"2024-04-25T13:11:06.564627+0000","last_fullsized":"2024-04-25T13:11:06.564627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:53.592529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565162+0000","last_change":"2024-04-25T13:11:05.279109+0000","last_active":"2024-04-25T13:11:06.565162+0000","last_peered":"2024-04-25T13:11:06.565162+0000","last_clean":"2024-04-25T13:11:06.565162+0000","last_became_active":"2024-04-25T13:11:05.278523+0000","last_became_peered":"2024-04-25T13:11:05.278523+0000","last_unstale":"2024-04-25T13:11:06.565162+0000","last_undegraded":"2024-04-25T13:11:06.565162+0000","last_fullsized":"2024-04-25T13:11:06.565162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:24.646025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.292953+0000","last_change":"2024-04-25T13:11:05.292039+0000","last_active":"2024-04-25T13:11:05.292953+0000","last_peered":"2024-04-25T13:11:05.292953+0000","last_clean":"2024-04-25T13:11:05.292953+0000","last_became_active":"2024-04-25T13:11:05.291649+0000","last_became_peered":"2024-04-25T13:11:05.291649+0000","last_unstale":"2024-04-25T13:11:05.292953+0000","last_undegraded":"2024-04-25T13:11:05.292953+0000","last_fullsized":"2024-04-25T13:11:05.292953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:35.392213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273124+0000","last_change":"2024-04-25T13:11:05.287067+0000","last_active":"2024-04-25T13:11:06.273124+0000","last_peered":"2024-04-25T13:11:06.273124+0000","last_clean":"2024-04-25T13:11:06.273124+0000","last_became_active":"2024-04-25T13:11:05.286664+0000","last_became_peered":"2024-04-25T13:11:05.286664+0000","last_unstale":"2024-04-25T13:11:06.273124+0000","last_undegraded":"2024-04-25T13:11:06.273124+0000","last_fullsized":"2024-04-25T13:11:06.273124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:12.677498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565670+0000","last_change":"2024-04-25T13:11:05.293192+0000","last_active":"2024-04-25T13:11:06.565670+0000","last_peered":"2024-04-25T13:11:06.565670+0000","last_clean":"2024-04-25T13:11:06.565670+0000","last_became_active":"2024-04-25T13:11:05.292663+0000","last_became_peered":"2024-04-25T13:11:05.292663+0000","last_unstale":"2024-04-25T13:11:06.565670+0000","last_undegraded":"2024-04-25T13:11:06.565670+0000","last_fullsized":"2024-04-25T13:11:06.565670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:14.942484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.566296+0000","last_change":"2024-04-25T13:11:05.278090+0000","last_active":"2024-04-25T13:11:06.566296+0000","last_peered":"2024-04-25T13:11:06.566296+0000","last_clean":"2024-04-25T13:11:06.566296+0000","last_became_active":"2024-04-25T13:11:05.277744+0000","last_became_peered":"2024-04-25T13:11:05.277744+0000","last_unstale":"2024-04-25T13:11:06.566296+0000","last_undegraded":"2024-04-25T13:11:06.566296+0000","last_fullsized":"2024-04-25T13:11:06.566296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:49.311241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564727+0000","last_change":"2024-04-25T13:11:05.277356+0000","last_active":"2024-04-25T13:11:06.564727+0000","last_peered":"2024-04-25T13:11:06.564727+0000","last_clean":"2024-04-25T13:11:06.564727+0000","last_became_active":"2024-04-25T13:11:05.276992+0000","last_became_peered":"2024-04-25T13:11:05.276992+0000","last_unstale":"2024-04-25T13:11:06.564727+0000","last_undegraded":"2024-04-25T13:11:06.564727+0000","last_fullsized":"2024-04-25T13:11:06.564727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:05.929613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564414+0000","last_change":"2024-04-25T13:11:05.271950+0000","last_active":"2024-04-25T13:11:06.564414+0000","last_peered":"2024-04-25T13:11:06.564414+0000","last_clean":"2024-04-25T13:11:06.564414+0000","last_became_active":"2024-04-25T13:11:05.271660+0000","last_became_peered":"2024-04-25T13:11:05.271660+0000","last_unstale":"2024-04-25T13:11:06.564414+0000","last_undegraded":"2024-04-25T13:11:06.564414+0000","last_fullsized":"2024-04-25T13:11:06.564414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:33.534854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564497+0000","last_change":"2024-04-25T13:11:05.283689+0000","last_active":"2024-04-25T13:11:06.564497+0000","last_peered":"2024-04-25T13:11:06.564497+0000","last_clean":"2024-04-25T13:11:06.564497+0000","last_became_active":"2024-04-25T13:11:05.283364+0000","last_became_peered":"2024-04-25T13:11:05.283364+0000","last_unstale":"2024-04-25T13:11:06.564497+0000","last_undegraded":"2024-04-25T13:11:06.564497+0000","last_fullsized":"2024-04-25T13:11:06.564497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:24.109350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273160+0000","last_change":"2024-04-25T13:11:05.280865+0000","last_active":"2024-04-25T13:11:06.273160+0000","last_peered":"2024-04-25T13:11:06.273160+0000","last_clean":"2024-04-25T13:11:06.273160+0000","last_became_active":"2024-04-25T13:11:05.280436+0000","last_became_peered":"2024-04-25T13:11:05.280436+0000","last_unstale":"2024-04-25T13:11:06.273160+0000","last_undegraded":"2024-04-25T13:11:06.273160+0000","last_fullsized":"2024-04-25T13:11:06.273160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:58.960578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272318+0000","last_change":"2024-04-25T13:11:05.282844+0000","last_active":"2024-04-25T13:11:06.272318+0000","last_peered":"2024-04-25T13:11:06.272318+0000","last_clean":"2024-04-25T13:11:06.272318+0000","last_became_active":"2024-04-25T13:11:05.282502+0000","last_became_peered":"2024-04-25T13:11:05.282502+0000","last_unstale":"2024-04-25T13:11:06.272318+0000","last_undegraded":"2024-04-25T13:11:06.272318+0000","last_fullsized":"2024-04-25T13:11:06.272318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:58.014902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564955+0000","last_change":"2024-04-25T13:11:05.276419+0000","last_active":"2024-04-25T13:11:06.564955+0000","last_peered":"2024-04-25T13:11:06.564955+0000","last_clean":"2024-04-25T13:11:06.564955+0000","last_became_active":"2024-04-25T13:11:05.275953+0000","last_became_peered":"2024-04-25T13:11:05.275953+0000","last_unstale":"2024-04-25T13:11:06.564955+0000","last_undegraded":"2024-04-25T13:11:06.564955+0000","last_fullsized":"2024-04-25T13:11:06.564955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:41.993632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.294580+0000","last_change":"2024-04-25T13:11:04.278345+0000","last_active":"2024-04-25T13:11:08.294580+0000","last_peered":"2024-04-25T13:11:08.294580+0000","last_clean":"2024-04-25T13:11:08.294580+0000","last_became_active":"2024-04-25T13:11:04.278017+0000","last_became_peered":"2024-04-25T13:11:04.278017+0000","last_unstale":"2024-04-25T13:11:08.294580+0000","last_undegraded":"2024-04-25T13:11:08.294580+0000","last_fullsized":"2024-04-25T13:11:08.294580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:34.496703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565116+0000","last_change":"2024-04-25T13:11:05.281069+0000","last_active":"2024-04-25T13:11:06.565116+0000","last_peered":"2024-04-25T13:11:06.565116+0000","last_clean":"2024-04-25T13:11:06.565116+0000","last_became_active":"2024-04-25T13:11:05.279428+0000","last_became_peered":"2024-04-25T13:11:05.279428+0000","last_unstale":"2024-04-25T13:11:06.565116+0000","last_undegraded":"2024-04-25T13:11:06.565116+0000","last_fullsized":"2024-04-25T13:11:06.565116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:53.224215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565213+0000","last_change":"2024-04-25T13:11:04.276017+0000","last_active":"2024-04-25T13:11:06.565213+0000","last_peered":"2024-04-25T13:11:06.565213+0000","last_clean":"2024-04-25T13:11:06.565213+0000","last_became_active":"2024-04-25T13:11:04.275718+0000","last_became_peered":"2024-04-25T13:11:04.275718+0000","last_unstale":"2024-04-25T13:11:06.565213+0000","last_undegraded":"2024-04-25T13:11:06.565213+0000","last_fullsized":"2024-04-25T13:11:06.565213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:45.850856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565503+0000","last_change":"2024-04-25T13:11:05.283185+0000","last_active":"2024-04-25T13:11:06.565503+0000","last_peered":"2024-04-25T13:11:06.565503+0000","last_clean":"2024-04-25T13:11:06.565503+0000","last_became_active":"2024-04-25T13:11:05.282876+0000","last_became_peered":"2024-04-25T13:11:05.282876+0000","last_unstale":"2024-04-25T13:11:06.565503+0000","last_undegraded":"2024-04-25T13:11:06.565503+0000","last_fullsized":"2024-04-25T13:11:06.565503+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:08.403124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.293012+0000","last_change":"2024-04-25T13:11:04.282188+0000","last_active":"2024-04-25T13:11:08.293012+0000","last_peered":"2024-04-25T13:11:08.293012+0000","last_clean":"2024-04-25T13:11:08.293012+0000","last_became_active":"2024-04-25T13:11:04.281569+0000","last_became_peered":"2024-04-25T13:11:04.281569+0000","last_unstale":"2024-04-25T13:11:08.293012+0000","last_undegraded":"2024-04-25T13:11:08.293012+0000","last_fullsized":"2024-04-25T13:11:08.293012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:45.502754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564612+0000","last_change":"2024-04-25T13:11:05.292858+0000","last_active":"2024-04-25T13:11:06.564612+0000","last_peered":"2024-04-25T13:11:06.564612+0000","last_clean":"2024-04-25T13:11:06.564612+0000","last_became_active":"2024-04-25T13:11:05.292474+0000","last_became_peered":"2024-04-25T13:11:05.292474+0000","last_unstale":"2024-04-25T13:11:06.564612+0000","last_undegraded":"2024-04-25T13:11:06.564612+0000","last_fullsized":"2024-04-25T13:11:06.564612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:46.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403943+0000","last_change":"2024-04-25T13:11:04.281104+0000","last_active":"2024-04-25T13:11:06.403943+0000","last_peered":"2024-04-25T13:11:06.403943+0000","last_clean":"2024-04-25T13:11:06.403943+0000","last_became_active":"2024-04-25T13:11:04.280644+0000","last_became_peered":"2024-04-25T13:11:04.280644+0000","last_unstale":"2024-04-25T13:11:06.403943+0000","last_undegraded":"2024-04-25T13:11:06.403943+0000","last_fullsized":"2024-04-25T13:11:06.403943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:15.938218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473117+0000","last_change":"2024-04-25T13:11:05.287532+0000","last_active":"2024-04-25T13:11:06.473117+0000","last_peered":"2024-04-25T13:11:06.473117+0000","last_clean":"2024-04-25T13:11:06.473117+0000","last_became_active":"2024-04-25T13:11:05.287225+0000","last_became_peered":"2024-04-25T13:11:05.287225+0000","last_unstale":"2024-04-25T13:11:06.473117+0000","last_undegraded":"2024-04-25T13:11:06.473117+0000","last_fullsized":"2024-04-25T13:11:06.473117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:23.531317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472773+0000","last_change":"2024-04-25T13:11:04.277567+0000","last_active":"2024-04-25T13:11:06.472773+0000","last_peered":"2024-04-25T13:11:06.472773+0000","last_clean":"2024-04-25T13:11:06.472773+0000","last_became_active":"2024-04-25T13:11:04.277098+0000","last_became_peered":"2024-04-25T13:11:04.277098+0000","last_unstale":"2024-04-25T13:11:06.472773+0000","last_undegraded":"2024-04-25T13:11:06.472773+0000","last_fullsized":"2024-04-25T13:11:06.472773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:23.592989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403672+0000","last_change":"2024-04-25T13:11:05.271586+0000","last_active":"2024-04-25T13:11:06.403672+0000","last_peered":"2024-04-25T13:11:06.403672+0000","last_clean":"2024-04-25T13:11:06.403672+0000","last_became_active":"2024-04-25T13:11:05.271233+0000","last_became_peered":"2024-04-25T13:11:05.271233+0000","last_unstale":"2024-04-25T13:11:06.403672+0000","last_undegraded":"2024-04-25T13:11:06.403672+0000","last_fullsized":"2024-04-25T13:11:06.403672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:53.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.294705+0000","last_change":"2024-04-25T13:11:04.283740+0000","last_active":"2024-04-25T13:11:05.294705+0000","last_peered":"2024-04-25T13:11:05.294705+0000","last_clean":"2024-04-25T13:11:05.294705+0000","last_became_active":"2024-04-25T13:11:04.283428+0000","last_became_peered":"2024-04-25T13:11:04.283428+0000","last_unstale":"2024-04-25T13:11:05.294705+0000","last_undegraded":"2024-04-25T13:11:05.294705+0000","last_fullsized":"2024-04-25T13:11:05.294705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:47.208036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473028+0000","last_change":"2024-04-25T13:11:05.281360+0000","last_active":"2024-04-25T13:11:06.473028+0000","last_peered":"2024-04-25T13:11:06.473028+0000","last_clean":"2024-04-25T13:11:06.473028+0000","last_became_active":"2024-04-25T13:11:05.281048+0000","last_became_peered":"2024-04-25T13:11:05.281048+0000","last_unstale":"2024-04-25T13:11:06.473028+0000","last_undegraded":"2024-04-25T13:11:06.473028+0000","last_fullsized":"2024-04-25T13:11:06.473028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:27.285125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564699+0000","last_change":"2024-04-25T13:11:04.272020+0000","last_active":"2024-04-25T13:11:06.564699+0000","last_peered":"2024-04-25T13:11:06.564699+0000","last_clean":"2024-04-25T13:11:06.564699+0000","last_became_active":"2024-04-25T13:11:04.271472+0000","last_became_peered":"2024-04-25T13:11:04.271472+0000","last_unstale":"2024-04-25T13:11:06.564699+0000","last_undegraded":"2024-04-25T13:11:06.564699+0000","last_fullsized":"2024-04-25T13:11:06.564699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:32.784185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564463+0000","last_change":"2024-04-25T13:11:05.272604+0000","last_active":"2024-04-25T13:11:06.564463+0000","last_peered":"2024-04-25T13:11:06.564463+0000","last_clean":"2024-04-25T13:11:06.564463+0000","last_became_active":"2024-04-25T13:11:05.272246+0000","last_became_peered":"2024-04-25T13:11:05.272246+0000","last_unstale":"2024-04-25T13:11:06.564463+0000","last_undegraded":"2024-04-25T13:11:06.564463+0000","last_fullsized":"2024-04-25T13:11:06.564463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:04.945342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472869+0000","last_change":"2024-04-25T13:11:04.279441+0000","last_active":"2024-04-25T13:11:06.472869+0000","last_peered":"2024-04-25T13:11:06.472869+0000","last_clean":"2024-04-25T13:11:06.472869+0000","last_became_active":"2024-04-25T13:11:04.278940+0000","last_became_peered":"2024-04-25T13:11:04.278940+0000","last_unstale":"2024-04-25T13:11:06.472869+0000","last_undegraded":"2024-04-25T13:11:06.472869+0000","last_fullsized":"2024-04-25T13:11:06.472869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:20.507906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272264+0000","last_change":"2024-04-25T13:11:05.282585+0000","last_active":"2024-04-25T13:11:06.272264+0000","last_peered":"2024-04-25T13:11:06.272264+0000","last_clean":"2024-04-25T13:11:06.272264+0000","last_became_active":"2024-04-25T13:11:05.282248+0000","last_became_peered":"2024-04-25T13:11:05.282248+0000","last_unstale":"2024-04-25T13:11:06.272264+0000","last_undegraded":"2024-04-25T13:11:06.272264+0000","last_fullsized":"2024-04-25T13:11:06.272264+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.671230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.274212+0000","last_change":"2024-04-25T13:11:04.277079+0000","last_active":"2024-04-25T13:11:05.274212+0000","last_peered":"2024-04-25T13:11:05.274212+0000","last_clean":"2024-04-25T13:11:05.274212+0000","last_became_active":"2024-04-25T13:11:04.276694+0000","last_became_peered":"2024-04-25T13:11:04.276694+0000","last_unstale":"2024-04-25T13:11:05.274212+0000","last_undegraded":"2024-04-25T13:11:05.274212+0000","last_fullsized":"2024-04-25T13:11:05.274212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:36.714106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565026+0000","last_change":"2024-04-25T13:11:05.292920+0000","last_active":"2024-04-25T13:11:06.565026+0000","last_peered":"2024-04-25T13:11:06.565026+0000","last_clean":"2024-04-25T13:11:06.565026+0000","last_became_active":"2024-04-25T13:11:05.292556+0000","last_became_peered":"2024-04-25T13:11:05.292556+0000","last_unstale":"2024-04-25T13:11:06.565026+0000","last_undegraded":"2024-04-25T13:11:06.565026+0000","last_fullsized":"2024-04-25T13:11:06.565026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:28.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.267065+0000","last_change":"2024-04-25T13:11:04.277566+0000","last_active":"2024-04-25T13:11:07.267065+0000","last_peered":"2024-04-25T13:11:07.267065+0000","last_clean":"2024-04-25T13:11:07.267065+0000","last_became_active":"2024-04-25T13:11:04.276909+0000","last_became_peered":"2024-04-25T13:11:04.276909+0000","last_unstale":"2024-04-25T13:11:07.267065+0000","last_undegraded":"2024-04-25T13:11:07.267065+0000","last_fullsized":"2024-04-25T13:11:07.267065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:43.907256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565559+0000","last_change":"2024-04-25T13:11:05.273851+0000","last_active":"2024-04-25T13:11:06.565559+0000","last_peered":"2024-04-25T13:11:06.565559+0000","last_clean":"2024-04-25T13:11:06.565559+0000","last_became_active":"2024-04-25T13:11:05.273503+0000","last_became_peered":"2024-04-25T13:11:05.273503+0000","last_unstale":"2024-04-25T13:11:06.565559+0000","last_undegraded":"2024-04-25T13:11:06.565559+0000","last_fullsized":"2024-04-25T13:11:06.565559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:38.532019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473082+0000","last_change":"2024-04-25T13:11:04.279446+0000","last_active":"2024-04-25T13:11:06.473082+0000","last_peered":"2024-04-25T13:11:06.473082+0000","last_clean":"2024-04-25T13:11:06.473082+0000","last_became_active":"2024-04-25T13:11:04.279083+0000","last_became_peered":"2024-04-25T13:11:04.279083+0000","last_unstale":"2024-04-25T13:11:06.473082+0000","last_undegraded":"2024-04-25T13:11:06.473082+0000","last_fullsized":"2024-04-25T13:11:06.473082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:28.719965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565139+0000","last_change":"2024-04-25T13:11:05.283795+0000","last_active":"2024-04-25T13:11:06.565139+0000","last_peered":"2024-04-25T13:11:06.565139+0000","last_clean":"2024-04-25T13:11:06.565139+0000","last_became_active":"2024-04-25T13:11:05.283356+0000","last_became_peered":"2024-04-25T13:11:05.283356+0000","last_unstale":"2024-04-25T13:11:06.565139+0000","last_undegraded":"2024-04-25T13:11:06.565139+0000","last_fullsized":"2024-04-25T13:11:06.565139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:57.095913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'6","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.298202+0000","last_change":"2024-04-25T13:11:04.277100+0000","last_active":"2024-04-25T13:11:05.298202+0000","last_peered":"2024-04-25T13:11:05.298202+0000","last_clean":"2024-04-25T13:11:05.298202+0000","last_became_active":"2024-04-25T13:11:04.276699+0000","last_became_peered":"2024-04-25T13:11:04.276699+0000","last_unstale":"2024-04-25T13:11:05.298202+0000","last_undegraded":"2024-04-25T13:11:05.298202+0000","last_fullsized":"2024-04-25T13:11:05.298202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:54.362024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403726+0000","last_change":"2024-04-25T13:11:05.284175+0000","last_active":"2024-04-25T13:11:06.403726+0000","last_peered":"2024-04-25T13:11:06.403726+0000","last_clean":"2024-04-25T13:11:06.403726+0000","last_became_active":"2024-04-25T13:11:05.283830+0000","last_became_peered":"2024-04-25T13:11:05.283830+0000","last_unstale":"2024-04-25T13:11:06.403726+0000","last_undegraded":"2024-04-25T13:11:06.403726+0000","last_fullsized":"2024-04-25T13:11:06.403726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:18.462117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.291196+0000","last_change":"2024-04-25T13:11:04.277374+0000","last_active":"2024-04-25T13:11:08.291196+0000","last_peered":"2024-04-25T13:11:08.291196+0000","last_clean":"2024-04-25T13:11:08.291196+0000","last_became_active":"2024-04-25T13:11:04.276995+0000","last_became_peered":"2024-04-25T13:11:04.276995+0000","last_unstale":"2024-04-25T13:11:08.291196+0000","last_undegraded":"2024-04-25T13:11:08.291196+0000","last_fullsized":"2024-04-25T13:11:08.291196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:04.856506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.566037+0000","last_change":"2024-04-25T13:11:05.284192+0000","last_active":"2024-04-25T13:11:06.566037+0000","last_peered":"2024-04-25T13:11:06.566037+0000","last_clean":"2024-04-25T13:11:06.566037+0000","last_became_active":"2024-04-25T13:11:05.283877+0000","last_became_peered":"2024-04-25T13:11:05.283877+0000","last_unstale":"2024-04-25T13:11:06.566037+0000","last_undegraded":"2024-04-25T13:11:06.566037+0000","last_fullsized":"2024-04-25T13:11:06.566037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:07.297999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472916+0000","last_change":"2024-04-25T13:11:04.276776+0000","last_active":"2024-04-25T13:11:06.472916+0000","last_peered":"2024-04-25T13:11:06.472916+0000","last_clean":"2024-04-25T13:11:06.472916+0000","last_became_active":"2024-04-25T13:11:04.276302+0000","last_became_peered":"2024-04-25T13:11:04.276302+0000","last_unstale":"2024-04-25T13:11:06.472916+0000","last_undegraded":"2024-04-25T13:11:06.472916+0000","last_fullsized":"2024-04-25T13:11:06.472916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:49.560839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273226+0000","last_change":"2024-04-25T13:11:05.281019+0000","last_active":"2024-04-25T13:11:06.273226+0000","last_peered":"2024-04-25T13:11:06.273226+0000","last_clean":"2024-04-25T13:11:06.273226+0000","last_became_active":"2024-04-25T13:11:05.280566+0000","last_became_peered":"2024-04-25T13:11:05.280566+0000","last_unstale":"2024-04-25T13:11:06.273226+0000","last_undegraded":"2024-04-25T13:11:06.273226+0000","last_fullsized":"2024-04-25T13:11:06.273226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:34.163825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:09.304146+0000","last_change":"2024-04-25T13:11:04.274968+0000","last_active":"2024-04-25T13:11:09.304146+0000","last_peered":"2024-04-25T13:11:09.304146+0000","last_clean":"2024-04-25T13:11:09.304146+0000","last_became_active":"2024-04-25T13:11:04.274617+0000","last_became_peered":"2024-04-25T13:11:04.274617+0000","last_unstale":"2024-04-25T13:11:09.304146+0000","last_undegraded":"2024-04-25T13:11:09.304146+0000","last_fullsized":"2024-04-25T13:11:09.304146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.502451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272999+0000","last_change":"2024-04-25T13:11:05.284103+0000","last_active":"2024-04-25T13:11:06.272999+0000","last_peered":"2024-04-25T13:11:06.272999+0000","last_clean":"2024-04-25T13:11:06.272999+0000","last_became_active":"2024-04-25T13:11:05.283689+0000","last_became_peered":"2024-04-25T13:11:05.283689+0000","last_unstale":"2024-04-25T13:11:06.272999+0000","last_undegraded":"2024-04-25T13:11:06.272999+0000","last_fullsized":"2024-04-25T13:11:06.272999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:09.967889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.295105+0000","last_change":"2024-04-25T13:11:04.281231+0000","last_active":"2024-04-25T13:11:08.295105+0000","last_peered":"2024-04-25T13:11:08.295105+0000","last_clean":"2024-04-25T13:11:08.295105+0000","last_became_active":"2024-04-25T13:11:04.280866+0000","last_became_peered":"2024-04-25T13:11:04.280866+0000","last_unstale":"2024-04-25T13:11:08.295105+0000","last_undegraded":"2024-04-25T13:11:08.295105+0000","last_fullsized":"2024-04-25T13:11:08.295105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:37.284138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472802+0000","last_change":"2024-04-25T13:11:05.283350+0000","last_active":"2024-04-25T13:11:06.472802+0000","last_peered":"2024-04-25T13:11:06.472802+0000","last_clean":"2024-04-25T13:11:06.472802+0000","last_became_active":"2024-04-25T13:11:05.282447+0000","last_became_peered":"2024-04-25T13:11:05.282447+0000","last_unstale":"2024-04-25T13:11:06.472802+0000","last_undegraded":"2024-04-25T13:11:06.472802+0000","last_fullsized":"2024-04-25T13:11:06.472802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:19.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403853+0000","last_change":"2024-04-25T13:11:04.275563+0000","last_active":"2024-04-25T13:11:06.403853+0000","last_peered":"2024-04-25T13:11:06.403853+0000","last_clean":"2024-04-25T13:11:06.403853+0000","last_became_active":"2024-04-25T13:11:04.275151+0000","last_became_peered":"2024-04-25T13:11:04.275151+0000","last_unstale":"2024-04-25T13:11:06.403853+0000","last_undegraded":"2024-04-25T13:11:06.403853+0000","last_fullsized":"2024-04-25T13:11:06.403853+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:27.629485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272933+0000","last_change":"2024-04-25T13:11:05.280723+0000","last_active":"2024-04-25T13:11:06.272933+0000","last_peered":"2024-04-25T13:11:06.272933+0000","last_clean":"2024-04-25T13:11:06.272933+0000","last_became_active":"2024-04-25T13:11:05.280271+0000","last_became_peered":"2024-04-25T13:11:05.280271+0000","last_unstale":"2024-04-25T13:11:06.272933+0000","last_undegraded":"2024-04-25T13:11:06.272933+0000","last_fullsized":"2024-04-25T13:11:06.272933+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:18.099610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272150+0000","last_change":"2024-04-25T13:11:04.275774+0000","last_active":"2024-04-25T13:11:06.272150+0000","last_peered":"2024-04-25T13:11:06.272150+0000","last_clean":"2024-04-25T13:11:06.272150+0000","last_became_active":"2024-04-25T13:11:04.275459+0000","last_became_peered":"2024-04-25T13:11:04.275459+0000","last_unstale":"2024-04-25T13:11:06.272150+0000","last_undegraded":"2024-04-25T13:11:06.272150+0000","last_fullsized":"2024-04-25T13:11:06.272150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.465906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.285696+0000","last_change":"2024-04-25T13:11:05.285237+0000","last_active":"2024-04-25T13:11:05.285696+0000","last_peered":"2024-04-25T13:11:05.285696+0000","last_clean":"2024-04-25T13:11:05.285696+0000","last_became_active":"2024-04-25T13:11:05.284875+0000","last_became_peered":"2024-04-25T13:11:05.284875+0000","last_unstale":"2024-04-25T13:11:05.285696+0000","last_undegraded":"2024-04-25T13:11:05.285696+0000","last_fullsized":"2024-04-25T13:11:05.285696+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:12.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272870+0000","last_change":"2024-04-25T13:11:04.273216+0000","last_active":"2024-04-25T13:11:06.272870+0000","last_peered":"2024-04-25T13:11:06.272870+0000","last_clean":"2024-04-25T13:11:06.272870+0000","last_became_active":"2024-04-25T13:11:04.272850+0000","last_became_peered":"2024-04-25T13:11:04.272850+0000","last_unstale":"2024-04-25T13:11:06.272870+0000","last_undegraded":"2024-04-25T13:11:06.272870+0000","last_fullsized":"2024-04-25T13:11:06.272870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:53.566971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:11:05.290761+0000","last_change":"2024-04-25T13:11:05.289951+0000","last_active":"2024-04-25T13:11:05.290761+0000","last_peered":"2024-04-25T13:11:05.290761+0000","last_clean":"2024-04-25T13:11:05.290761+0000","last_became_active":"2024-04-25T13:11:05.289575+0000","last_became_peered":"2024-04-25T13:11:05.289575+0000","last_unstale":"2024-04-25T13:11:05.290761+0000","last_undegraded":"2024-04-25T13:11:05.290761+0000","last_fullsized":"2024-04-25T13:11:05.290761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:33.843458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473196+0000","last_change":"2024-04-25T13:11:05.278811+0000","last_active":"2024-04-25T13:11:06.473196+0000","last_peered":"2024-04-25T13:11:06.473196+0000","last_clean":"2024-04-25T13:11:06.473196+0000","last_became_active":"2024-04-25T13:11:05.278476+0000","last_became_peered":"2024-04-25T13:11:05.278476+0000","last_unstale":"2024-04-25T13:11:06.473196+0000","last_undegraded":"2024-04-25T13:11:06.473196+0000","last_fullsized":"2024-04-25T13:11:06.473196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:40.656046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564652+0000","last_change":"2024-04-25T13:11:04.277793+0000","last_active":"2024-04-25T13:11:06.564652+0000","last_peered":"2024-04-25T13:11:06.564652+0000","last_clean":"2024-04-25T13:11:06.564652+0000","last_became_active":"2024-04-25T13:11:04.277254+0000","last_became_peered":"2024-04-25T13:11:04.277254+0000","last_unstale":"2024-04-25T13:11:06.564652+0000","last_undegraded":"2024-04-25T13:11:06.564652+0000","last_fullsized":"2024-04-25T13:11:06.564652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:19.481074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564832+0000","last_change":"2024-04-25T13:11:05.285358+0000","last_active":"2024-04-25T13:11:06.564832+0000","last_peered":"2024-04-25T13:11:06.564832+0000","last_clean":"2024-04-25T13:11:06.564832+0000","last_became_active":"2024-04-25T13:11:05.284641+0000","last_became_peered":"2024-04-25T13:11:05.284641+0000","last_unstale":"2024-04-25T13:11:06.564832+0000","last_undegraded":"2024-04-25T13:11:06.564832+0000","last_fullsized":"2024-04-25T13:11:06.564832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:33.245556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:09.300025+0000","last_change":"2024-04-25T13:11:04.272294+0000","last_active":"2024-04-25T13:11:09.300025+0000","last_peered":"2024-04-25T13:11:09.300025+0000","last_clean":"2024-04-25T13:11:09.300025+0000","last_became_active":"2024-04-25T13:11:04.271898+0000","last_became_peered":"2024-04-25T13:11:04.271898+0000","last_unstale":"2024-04-25T13:11:09.300025+0000","last_undegraded":"2024-04-25T13:11:09.300025+0000","last_fullsized":"2024-04-25T13:11:09.300025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:29.130773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273010+0000","last_change":"2024-04-25T13:09:54.520980+0000","last_active":"2024-04-25T13:11:06.273010+0000","last_peered":"2024-04-25T13:11:06.273010+0000","last_clean":"2024-04-25T13:11:06.273010+0000","last_became_active":"2024-04-25T13:09:54.520446+0000","last_became_peered":"2024-04-25T13:09:54.520446+0000","last_unstale":"2024-04-25T13:11:06.273010+0000","last_undegraded":"2024-04-25T13:11:06.273010+0000","last_fullsized":"2024-04-25T13:11:06.273010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:06:33.347730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:06:33.347730+0000","last_clean_scrub_stamp":"2024-04-25T13:06:33.347730+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:00.996641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564588+0000","last_change":"2024-04-25T13:11:05.279449+0000","last_active":"2024-04-25T13:11:06.564588+0000","last_peered":"2024-04-25T13:11:06.564588+0000","last_clean":"2024-04-25T13:11:06.564588+0000","last_became_active":"2024-04-25T13:11:05.278982+0000","last_became_peered":"2024-04-25T13:11:05.278982+0000","last_unstale":"2024-04-25T13:11:06.564588+0000","last_undegraded":"2024-04-25T13:11:06.564588+0000","last_fullsized":"2024-04-25T13:11:06.564588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:44.647967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.293696+0000","last_change":"2024-04-25T13:11:04.279131+0000","last_active":"2024-04-25T13:11:08.293696+0000","last_peered":"2024-04-25T13:11:08.293696+0000","last_clean":"2024-04-25T13:11:08.293696+0000","last_became_active":"2024-04-25T13:11:04.278801+0000","last_became_peered":"2024-04-25T13:11:04.278801+0000","last_unstale":"2024-04-25T13:11:08.293696+0000","last_undegraded":"2024-04-25T13:11:08.293696+0000","last_fullsized":"2024-04-25T13:11:08.293696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:06.341497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.265750+0000","last_change":"2024-04-25T13:11:04.275784+0000","last_active":"2024-04-25T13:11:07.265750+0000","last_peered":"2024-04-25T13:11:07.265750+0000","last_clean":"2024-04-25T13:11:07.265750+0000","last_became_active":"2024-04-25T13:11:04.275487+0000","last_became_peered":"2024-04-25T13:11:04.275487+0000","last_unstale":"2024-04-25T13:11:07.265750+0000","last_undegraded":"2024-04-25T13:11:07.265750+0000","last_fullsized":"2024-04-25T13:11:07.265750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.405056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565627+0000","last_change":"2024-04-25T13:11:05.275854+0000","last_active":"2024-04-25T13:11:06.565627+0000","last_peered":"2024-04-25T13:11:06.565627+0000","last_clean":"2024-04-25T13:11:06.565627+0000","last_became_active":"2024-04-25T13:11:05.275527+0000","last_became_peered":"2024-04-25T13:11:05.275527+0000","last_unstale":"2024-04-25T13:11:06.565627+0000","last_undegraded":"2024-04-25T13:11:06.565627+0000","last_fullsized":"2024-04-25T13:11:06.565627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:42.300458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.294489+0000","last_change":"2024-04-25T13:11:04.279415+0000","last_active":"2024-04-25T13:11:08.294489+0000","last_peered":"2024-04-25T13:11:08.294489+0000","last_clean":"2024-04-25T13:11:08.294489+0000","last_became_active":"2024-04-25T13:11:04.279119+0000","last_became_peered":"2024-04-25T13:11:04.279119+0000","last_unstale":"2024-04-25T13:11:08.294489+0000","last_undegraded":"2024-04-25T13:11:08.294489+0000","last_fullsized":"2024-04-25T13:11:08.294489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:18.917509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565327+0000","last_change":"2024-04-25T13:11:05.276317+0000","last_active":"2024-04-25T13:11:06.565327+0000","last_peered":"2024-04-25T13:11:06.565327+0000","last_clean":"2024-04-25T13:11:06.565327+0000","last_became_active":"2024-04-25T13:11:05.275781+0000","last_became_peered":"2024-04-25T13:11:05.275781+0000","last_unstale":"2024-04-25T13:11:06.565327+0000","last_undegraded":"2024-04-25T13:11:06.565327+0000","last_fullsized":"2024-04-25T13:11:06.565327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:41.628388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.265929+0000","last_change":"2024-04-25T13:11:04.273617+0000","last_active":"2024-04-25T13:11:07.265929+0000","last_peered":"2024-04-25T13:11:07.265929+0000","last_clean":"2024-04-25T13:11:07.265929+0000","last_became_active":"2024-04-25T13:11:04.272749+0000","last_became_peered":"2024-04-25T13:11:04.272749+0000","last_unstale":"2024-04-25T13:11:07.265929+0000","last_undegraded":"2024-04-25T13:11:07.265929+0000","last_fullsized":"2024-04-25T13:11:07.265929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:14.484215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565450+0000","last_change":"2024-04-25T13:11:05.274423+0000","last_active":"2024-04-25T13:11:06.565450+0000","last_peered":"2024-04-25T13:11:06.565450+0000","last_clean":"2024-04-25T13:11:06.565450+0000","last_became_active":"2024-04-25T13:11:05.273180+0000","last_became_peered":"2024-04-25T13:11:05.273180+0000","last_unstale":"2024-04-25T13:11:06.565450+0000","last_undegraded":"2024-04-25T13:11:06.565450+0000","last_fullsized":"2024-04-25T13:11:06.565450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:51.082000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.266877+0000","last_change":"2024-04-25T13:11:04.275671+0000","last_active":"2024-04-25T13:11:07.266877+0000","last_peered":"2024-04-25T13:11:07.266877+0000","last_clean":"2024-04-25T13:11:07.266877+0000","last_became_active":"2024-04-25T13:11:04.275357+0000","last_became_peered":"2024-04-25T13:11:04.275357+0000","last_unstale":"2024-04-25T13:11:07.266877+0000","last_undegraded":"2024-04-25T13:11:07.266877+0000","last_fullsized":"2024-04-25T13:11:07.266877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:55.552043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.403484+0000","last_change":"2024-04-25T13:11:05.282327+0000","last_active":"2024-04-25T13:11:06.403484+0000","last_peered":"2024-04-25T13:11:06.403484+0000","last_clean":"2024-04-25T13:11:06.403484+0000","last_became_active":"2024-04-25T13:11:05.281903+0000","last_became_peered":"2024-04-25T13:11:05.281903+0000","last_unstale":"2024-04-25T13:11:06.403484+0000","last_undegraded":"2024-04-25T13:11:06.403484+0000","last_fullsized":"2024-04-25T13:11:06.403484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:40.490726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564738+0000","last_change":"2024-04-25T13:11:04.273065+0000","last_active":"2024-04-25T13:11:06.564738+0000","last_peered":"2024-04-25T13:11:06.564738+0000","last_clean":"2024-04-25T13:11:06.564738+0000","last_became_active":"2024-04-25T13:11:04.272723+0000","last_became_peered":"2024-04-25T13:11:04.272723+0000","last_unstale":"2024-04-25T13:11:06.564738+0000","last_undegraded":"2024-04-25T13:11:06.564738+0000","last_fullsized":"2024-04-25T13:11:06.564738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:56.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272593+0000","last_change":"2024-04-25T13:11:05.276845+0000","last_active":"2024-04-25T13:11:06.272593+0000","last_peered":"2024-04-25T13:11:06.272593+0000","last_clean":"2024-04-25T13:11:06.272593+0000","last_became_active":"2024-04-25T13:11:05.276466+0000","last_became_peered":"2024-04-25T13:11:05.276466+0000","last_unstale":"2024-04-25T13:11:06.272593+0000","last_undegraded":"2024-04-25T13:11:06.272593+0000","last_fullsized":"2024-04-25T13:11:06.272593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:55.934335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564489+0000","last_change":"2024-04-25T13:11:04.278970+0000","last_active":"2024-04-25T13:11:06.564489+0000","last_peered":"2024-04-25T13:11:06.564489+0000","last_clean":"2024-04-25T13:11:06.564489+0000","last_became_active":"2024-04-25T13:11:04.278594+0000","last_became_peered":"2024-04-25T13:11:04.278594+0000","last_unstale":"2024-04-25T13:11:06.564489+0000","last_undegraded":"2024-04-25T13:11:06.564489+0000","last_fullsized":"2024-04-25T13:11:06.564489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:31.099566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565810+0000","last_change":"2024-04-25T13:11:05.281008+0000","last_active":"2024-04-25T13:11:06.565810+0000","last_peered":"2024-04-25T13:11:06.565810+0000","last_clean":"2024-04-25T13:11:06.565810+0000","last_became_active":"2024-04-25T13:11:05.280581+0000","last_became_peered":"2024-04-25T13:11:05.280581+0000","last_unstale":"2024-04-25T13:11:06.565810+0000","last_undegraded":"2024-04-25T13:11:06.565810+0000","last_fullsized":"2024-04-25T13:11:06.565810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:28.636608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:07.267053+0000","last_change":"2024-04-25T13:11:04.281969+0000","last_active":"2024-04-25T13:11:07.267053+0000","last_peered":"2024-04-25T13:11:07.267053+0000","last_clean":"2024-04-25T13:11:07.267053+0000","last_became_active":"2024-04-25T13:11:04.281501+0000","last_became_peered":"2024-04-25T13:11:04.281501+0000","last_unstale":"2024-04-25T13:11:07.267053+0000","last_undegraded":"2024-04-25T13:11:07.267053+0000","last_fullsized":"2024-04-25T13:11:07.267053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:36.494799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273356+0000","last_change":"2024-04-25T13:11:05.288121+0000","last_active":"2024-04-25T13:11:06.273356+0000","last_peered":"2024-04-25T13:11:06.273356+0000","last_clean":"2024-04-25T13:11:06.273356+0000","last_became_active":"2024-04-25T13:11:05.287816+0000","last_became_peered":"2024-04-25T13:11:05.287816+0000","last_unstale":"2024-04-25T13:11:06.273356+0000","last_undegraded":"2024-04-25T13:11:06.273356+0000","last_fullsized":"2024-04-25T13:11:06.273356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:02.855646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:08.294750+0000","last_change":"2024-04-25T13:11:04.279883+0000","last_active":"2024-04-25T13:11:08.294750+0000","last_peered":"2024-04-25T13:11:08.294750+0000","last_clean":"2024-04-25T13:11:08.294750+0000","last_became_active":"2024-04-25T13:11:04.279408+0000","last_became_peered":"2024-04-25T13:11:04.279408+0000","last_unstale":"2024-04-25T13:11:08.294750+0000","last_undegraded":"2024-04-25T13:11:08.294750+0000","last_fullsized":"2024-04-25T13:11:08.294750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:42.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273320+0000","last_change":"2024-04-25T13:11:05.284845+0000","last_active":"2024-04-25T13:11:06.273320+0000","last_peered":"2024-04-25T13:11:06.273320+0000","last_clean":"2024-04-25T13:11:06.273320+0000","last_became_active":"2024-04-25T13:11:05.284300+0000","last_became_peered":"2024-04-25T13:11:05.284300+0000","last_unstale":"2024-04-25T13:11:06.273320+0000","last_undegraded":"2024-04-25T13:11:06.273320+0000","last_fullsized":"2024-04-25T13:11:06.273320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:54.233049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.472954+0000","last_change":"2024-04-25T13:11:04.274075+0000","last_active":"2024-04-25T13:11:06.472954+0000","last_peered":"2024-04-25T13:11:06.472954+0000","last_clean":"2024-04-25T13:11:06.472954+0000","last_became_active":"2024-04-25T13:11:04.273624+0000","last_became_peered":"2024-04-25T13:11:04.273624+0000","last_unstale":"2024-04-25T13:11:06.472954+0000","last_undegraded":"2024-04-25T13:11:06.472954+0000","last_fullsized":"2024-04-25T13:11:06.472954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:52.661939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273014+0000","last_change":"2024-04-25T13:11:05.283181+0000","last_active":"2024-04-25T13:11:06.273014+0000","last_peered":"2024-04-25T13:11:06.273014+0000","last_clean":"2024-04-25T13:11:06.273014+0000","last_became_active":"2024-04-25T13:11:05.282848+0000","last_became_peered":"2024-04-25T13:11:05.282848+0000","last_unstale":"2024-04-25T13:11:06.273014+0000","last_undegraded":"2024-04-25T13:11:06.273014+0000","last_fullsized":"2024-04-25T13:11:06.273014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:21.372069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565654+0000","last_change":"2024-04-25T13:11:04.270691+0000","last_active":"2024-04-25T13:11:06.565654+0000","last_peered":"2024-04-25T13:11:06.565654+0000","last_clean":"2024-04-25T13:11:06.565654+0000","last_became_active":"2024-04-25T13:11:04.270337+0000","last_became_peered":"2024-04-25T13:11:04.270337+0000","last_unstale":"2024-04-25T13:11:06.565654+0000","last_undegraded":"2024-04-25T13:11:06.565654+0000","last_fullsized":"2024-04-25T13:11:06.565654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:05.632472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.273400+0000","last_change":"2024-04-25T13:11:05.284284+0000","last_active":"2024-04-25T13:11:06.273400+0000","last_peered":"2024-04-25T13:11:06.273400+0000","last_clean":"2024-04-25T13:11:06.273400+0000","last_became_active":"2024-04-25T13:11:05.283952+0000","last_became_peered":"2024-04-25T13:11:05.283952+0000","last_unstale":"2024-04-25T13:11:06.273400+0000","last_undegraded":"2024-04-25T13:11:06.273400+0000","last_fullsized":"2024-04-25T13:11:06.273400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:32.396183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272662+0000","last_change":"2024-04-25T13:11:05.278064+0000","last_active":"2024-04-25T13:11:06.272662+0000","last_peered":"2024-04-25T13:11:06.272662+0000","last_clean":"2024-04-25T13:11:06.272662+0000","last_became_active":"2024-04-25T13:11:05.277680+0000","last_became_peered":"2024-04-25T13:11:05.277680+0000","last_unstale":"2024-04-25T13:11:06.272662+0000","last_undegraded":"2024-04-25T13:11:06.272662+0000","last_fullsized":"2024-04-25T13:11:06.272662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:32.048667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564662+0000","last_change":"2024-04-25T13:11:05.278432+0000","last_active":"2024-04-25T13:11:06.564662+0000","last_peered":"2024-04-25T13:11:06.564662+0000","last_clean":"2024-04-25T13:11:06.564662+0000","last_became_active":"2024-04-25T13:11:05.278089+0000","last_became_peered":"2024-04-25T13:11:05.278089+0000","last_unstale":"2024-04-25T13:11:06.564662+0000","last_undegraded":"2024-04-25T13:11:06.564662+0000","last_fullsized":"2024-04-25T13:11:06.564662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:49.505681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564676+0000","last_change":"2024-04-25T13:11:05.278072+0000","last_active":"2024-04-25T13:11:06.564676+0000","last_peered":"2024-04-25T13:11:06.564676+0000","last_clean":"2024-04-25T13:11:06.564676+0000","last_became_active":"2024-04-25T13:11:05.277720+0000","last_became_peered":"2024-04-25T13:11:05.277720+0000","last_unstale":"2024-04-25T13:11:06.564676+0000","last_undegraded":"2024-04-25T13:11:06.564676+0000","last_fullsized":"2024-04-25T13:11:06.564676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:14.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564955+0000","last_change":"2024-04-25T13:11:05.272611+0000","last_active":"2024-04-25T13:11:06.564955+0000","last_peered":"2024-04-25T13:11:06.564955+0000","last_clean":"2024-04-25T13:11:06.564955+0000","last_became_active":"2024-04-25T13:11:05.272318+0000","last_became_peered":"2024-04-25T13:11:05.272318+0000","last_unstale":"2024-04-25T13:11:06.564955+0000","last_undegraded":"2024-04-25T13:11:06.564955+0000","last_fullsized":"2024-04-25T13:11:06.564955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:52.796827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565769+0000","last_change":"2024-04-25T13:11:05.281146+0000","last_active":"2024-04-25T13:11:06.565769+0000","last_peered":"2024-04-25T13:11:06.565769+0000","last_clean":"2024-04-25T13:11:06.565769+0000","last_became_active":"2024-04-25T13:11:05.280897+0000","last_became_peered":"2024-04-25T13:11:05.280897+0000","last_unstale":"2024-04-25T13:11:06.565769+0000","last_undegraded":"2024-04-25T13:11:06.565769+0000","last_fullsized":"2024-04-25T13:11:06.565769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:27.348411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272877+0000","last_change":"2024-04-25T13:11:05.275482+0000","last_active":"2024-04-25T13:11:06.272877+0000","last_peered":"2024-04-25T13:11:06.272877+0000","last_clean":"2024-04-25T13:11:06.272877+0000","last_became_active":"2024-04-25T13:11:05.275066+0000","last_became_peered":"2024-04-25T13:11:05.275066+0000","last_unstale":"2024-04-25T13:11:06.272877+0000","last_undegraded":"2024-04-25T13:11:06.272877+0000","last_fullsized":"2024-04-25T13:11:06.272877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:27.286105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565058+0000","last_change":"2024-04-25T13:11:05.287931+0000","last_active":"2024-04-25T13:11:06.565058+0000","last_peered":"2024-04-25T13:11:06.565058+0000","last_clean":"2024-04-25T13:11:06.565058+0000","last_became_active":"2024-04-25T13:11:05.287540+0000","last_became_peered":"2024-04-25T13:11:05.287540+0000","last_unstale":"2024-04-25T13:11:06.565058+0000","last_undegraded":"2024-04-25T13:11:06.565058+0000","last_fullsized":"2024-04-25T13:11:06.565058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:16.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.565105+0000","last_change":"2024-04-25T13:11:05.274197+0000","last_active":"2024-04-25T13:11:06.565105+0000","last_peered":"2024-04-25T13:11:06.565105+0000","last_clean":"2024-04-25T13:11:06.565105+0000","last_became_active":"2024-04-25T13:11:05.273236+0000","last_became_peered":"2024-04-25T13:11:05.273236+0000","last_unstale":"2024-04-25T13:11:06.565105+0000","last_undegraded":"2024-04-25T13:11:06.565105+0000","last_fullsized":"2024-04-25T13:11:06.565105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:55.471002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.473103+0000","last_change":"2024-04-25T13:11:05.282693+0000","last_active":"2024-04-25T13:11:06.473103+0000","last_peered":"2024-04-25T13:11:06.473103+0000","last_clean":"2024-04-25T13:11:06.473103+0000","last_became_active":"2024-04-25T13:11:05.280002+0000","last_became_peered":"2024-04-25T13:11:05.280002+0000","last_unstale":"2024-04-25T13:11:06.473103+0000","last_undegraded":"2024-04-25T13:11:06.473103+0000","last_fullsized":"2024-04-25T13:11:06.473103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:08.661270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564671+0000","last_change":"2024-04-25T13:11:05.274581+0000","last_active":"2024-04-25T13:11:06.564671+0000","last_peered":"2024-04-25T13:11:06.564671+0000","last_clean":"2024-04-25T13:11:06.564671+0000","last_became_active":"2024-04-25T13:11:05.273634+0000","last_became_peered":"2024-04-25T13:11:05.273634+0000","last_unstale":"2024-04-25T13:11:06.564671+0000","last_undegraded":"2024-04-25T13:11:06.564671+0000","last_fullsized":"2024-04-25T13:11:06.564671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:13.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.564786+0000","last_change":"2024-04-25T13:11:05.284581+0000","last_active":"2024-04-25T13:11:06.564786+0000","last_peered":"2024-04-25T13:11:06.564786+0000","last_clean":"2024-04-25T13:11:06.564786+0000","last_became_active":"2024-04-25T13:11:05.283032+0000","last_became_peered":"2024-04-25T13:11:05.283032+0000","last_unstale":"2024-04-25T13:11:06.564786+0000","last_undegraded":"2024-04-25T13:11:06.564786+0000","last_fullsized":"2024-04-25T13:11:06.564786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:12.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.566245+0000","last_change":"2024-04-25T13:11:05.285081+0000","last_active":"2024-04-25T13:11:06.566245+0000","last_peered":"2024-04-25T13:11:06.566245+0000","last_clean":"2024-04-25T13:11:06.566245+0000","last_became_active":"2024-04-25T13:11:05.283717+0000","last_became_peered":"2024-04-25T13:11:05.283717+0000","last_unstale":"2024-04-25T13:11:06.566245+0000","last_undegraded":"2024-04-25T13:11:06.566245+0000","last_fullsized":"2024-04-25T13:11:06.566245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:46.381684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:11:06.272668+0000","last_change":"2024-04-25T13:11:05.278362+0000","last_active":"2024-04-25T13:11:06.272668+0000","last_peered":"2024-04-25T13:11:06.272668+0000","last_clean":"2024-04-25T13:11:06.272668+0000","last_became_active":"2024-04-25T13:11:05.277985+0000","last_became_peered":"2024-04-25T13:11:05.277985+0000","last_unstale":"2024-04-25T13:11:06.272668+0000","last_undegraded":"2024-04-25T13:11:06.272668+0000","last_fullsized":"2024-04-25T13:11:06.272668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:45.142805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3748,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":98,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":19709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":456042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1556480,"data_stored":1039064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743447,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":456278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":456198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1047620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":446890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":463672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":456938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":457728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":455564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920816,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36848,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708304,"statfs":{"total":95995035648,"available":95957303296,"internally_reserved":0,"allocated":1544192,"data_stored":1037440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:11:12.345 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:11:12.346 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T13:11:12.346 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T13:11:12.616 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:12.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:12 smithi193 ceph-mon[32743]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:12.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:12 smithi193 ceph-mon[32743]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:12.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:12 smithi193 ceph-mon[32743]: pgmap v290: 97 pgs: 97 active+clean; 580 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.8 KiB/s wr, 16 op/s 2024-04-25T13:11:12.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:12 smithi193 ceph-mon[32743]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:11:12.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:12 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1788884763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:11:12.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:12 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/665586143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:12.737 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:12 smithi072 ceph-mon[29199]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:12.737 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:12 smithi072 ceph-mon[29199]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:12.737 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:12 smithi072 ceph-mon[29199]: pgmap v290: 97 pgs: 97 active+clean; 580 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.8 KiB/s wr, 16 op/s 2024-04-25T13:11:12.737 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:12 smithi072 ceph-mon[29199]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:11:12.738 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:12 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1788884763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:11:12.738 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:12 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/665586143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:12.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:12 smithi164 ceph-mon[32418]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:12.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:12 smithi164 ceph-mon[32418]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:12.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:12 smithi164 ceph-mon[32418]: pgmap v290: 97 pgs: 97 active+clean; 580 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.8 KiB/s wr, 16 op/s 2024-04-25T13:11:12.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:12 smithi164 ceph-mon[32418]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:11:12.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:12 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1788884763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:11:12.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:12 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/665586143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:13.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:13 smithi193 ceph-mon[32743]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:11:13.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:13 smithi193 ceph-mon[32743]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:11:13.762 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:13 smithi072 ceph-mon[29199]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:11:13.762 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:13 smithi072 ceph-mon[29199]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:11:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:13 smithi164 ceph-mon[32418]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:11:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:13 smithi164 ceph-mon[32418]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:11:14.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2627407603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8104660 msgr2=0x7f4cc810af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2627407603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8104660 0x7f4cc810af10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000b0d0 tx=0x7f4cc000b3e0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2627407603 shutdown_connections 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2627407603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8104660 0x7f4cc810af10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2627407603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cc8102ac0 0x7f4cc8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2627407603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc8103cc0 0x7f4cc8104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2627407603 >> 172.21.15.72:0/2627407603 conn(0x7f4cc80fe060 msgr2=0x7f4cc81004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2627407603 shutdown_connections 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2627407603 wait complete. 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 Processor -- start 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.400+0000 7f4ccf51d700 1 -- start start 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccf51d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc8102ac0 0x7f4cc80767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:14.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccf51d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8103cc0 0x7f4cc8074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccf51d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cc8104660 0x7f4cc8075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccf51d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810e650 con 0x7f4cc8103cc0 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccf51d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810e4d0 con 0x7f4cc8102ac0 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccf51d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810e7d0 con 0x7f4cc8104660 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cccab8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8103cc0 0x7f4cc8074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cccab8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8103cc0 0x7f4cc8074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56476/0 (socket says 172.21.15.72:56476) 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cccab8700 1 -- 172.21.15.72:0/2438852060 learned_addr learned my addr 172.21.15.72:0/2438852060 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccdaba700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cc8104660 0x7f4cc8075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccd2b9700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc8102ac0 0x7f4cc80767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cccab8700 1 -- 172.21.15.72:0/2438852060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cc8104660 msgr2=0x7f4cc8075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cccab8700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cc8104660 0x7f4cc8075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cccab8700 1 -- 172.21.15.72:0/2438852060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc8102ac0 msgr2=0x7f4cc80767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:14.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cccab8700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc8102ac0 0x7f4cc80767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cccab8700 1 -- 172.21.15.72:0/2438852060 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc8075c20 con 0x7f4cc8103cc0 2024-04-25T13:11:14.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccd2b9700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc8102ac0 0x7f4cc80767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:11:14.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cccab8700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8103cc0 0x7f4cc8074e30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800b530 tx=0x7f4cb800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:14.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cbe7fc700 1 -- 172.21.15.72:0/2438852060 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8015020 con 0x7f4cc8103cc0 2024-04-25T13:11:14.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc81a2da0 con 0x7f4cc8103cc0 2024-04-25T13:11:14.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cbe7fc700 1 -- 172.21.15.72:0/2438852060 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb8011ce0 con 0x7f4cc8103cc0 2024-04-25T13:11:14.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4cbe7fc700 1 -- 172.21.15.72:0/2438852060 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8010e40 con 0x7f4cc8103cc0 2024-04-25T13:11:14.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.401+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc81a3280 con 0x7f4cc8103cc0 2024-04-25T13:11:14.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.402+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc802df80 con 0x7f4cc8103cc0 2024-04-25T13:11:14.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.403+0000 7f4cbe7fc700 1 -- 172.21.15.72:0/2438852060 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4cb8018020 con 0x7f4cc8103cc0 2024-04-25T13:11:14.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.404+0000 7f4cbe7fc700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4cb40a3d80 0x7f4cb40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:14.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.404+0000 7f4ccd2b9700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4cb40a3d80 0x7f4cb40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:14.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.404+0000 7f4cbe7fc700 1 -- 172.21.15.72:0/2438852060 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f4cb80cfb00 con 0x7f4cc8103cc0 2024-04-25T13:11:14.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.404+0000 7f4ccd2b9700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4cb40a3d80 0x7f4cb40a6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8103b20 tx=0x7f4cc4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:14.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.409+0000 7f4cbe7fc700 1 -- 172.21.15.72:0/2438852060 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cb801f020 con 0x7f4cc8103cc0 2024-04-25T13:11:14.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.686+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f4cc802e350 con 0x7f4cc8103cc0 2024-04-25T13:11:14.686 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:14 smithi072 ceph-mon[29199]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.9 KiB/s wr, 21 op/s 2024-04-25T13:11:14.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.688+0000 7f4cbe7fc700 1 -- 172.21.15.72:0/2438852060 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f4cb809c640 con 0x7f4cc8103cc0 2024-04-25T13:11:14.688 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2024-04-25T13:11:14.689 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQByVipmm3QFKRAAuxSDoV6cvLBwtBy7UABxkA== 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.691+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4cb40a3d80 msgr2=0x7f4cb40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.691+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4cb40a3d80 0x7f4cb40a6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8103b20 tx=0x7f4cc4008040 comp rx=0 tx=0).stop 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8103cc0 msgr2=0x7f4cc8074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8103cc0 0x7f4cc8074e30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800b530 tx=0x7f4cb800b840 comp rx=0 tx=0).stop 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 shutdown_connections 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cc8104660 0x7f4cc8075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc8102ac0 0x7f4cc80767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4cb40a3d80 0x7f4cb40a6240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 --2- 172.21.15.72:0/2438852060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cc8103cc0 0x7f4cc8074e30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 >> 172.21.15.72:0/2438852060 conn(0x7f4cc80fe060 msgr2=0x7f4cc8106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 shutdown_connections 2024-04-25T13:11:14.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:14.692+0000 7f4ccf51d700 1 -- 172.21.15.72:0/2438852060 wait complete. 2024-04-25T13:11:14.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:14 smithi193 ceph-mon[32743]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.9 KiB/s wr, 21 op/s 2024-04-25T13:11:14.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:14 smithi164 ceph-mon[32418]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.9 KiB/s wr, 21 op/s 2024-04-25T13:11:15.360 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:11:15.360 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T13:11:15.360 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T13:11:15.396 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T13:11:15.396 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T13:11:15.397 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T13:11:15.673 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:15.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:15 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2438852060' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:11:15.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:15 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2438852060' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:11:15.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:15 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2438852060' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:11:15.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:15 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2438852060' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:11:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:15 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2438852060' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:11:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:15 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2438852060' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:11:16.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:16 smithi072 ceph-mon[29199]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-25T13:11:16.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:16 smithi193 ceph-mon[32743]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-25T13:11:16.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:16 smithi164 ceph-mon[32418]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-25T13:11:17.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 -- 172.21.15.72:0/404962854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 msgr2=0x7f3d14102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:17.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/404962854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 0x7f3d14102f10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009990 tx=0x7f3d08009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 -- 172.21.15.72:0/404962854 shutdown_connections 2024-04-25T13:11:17.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/404962854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d14103450 0x7f3d1410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/404962854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d14108a50 0x7f3d14108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/404962854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 0x7f3d14102f10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 -- 172.21.15.72:0/404962854 >> 172.21.15.72:0/404962854 conn(0x7f3d14079ba0 msgr2=0x7f3d14079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:17.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 -- 172.21.15.72:0/404962854 shutdown_connections 2024-04-25T13:11:17.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 -- 172.21.15.72:0/404962854 wait complete. 2024-04-25T13:11:17.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 Processor -- start 2024-04-25T13:11:17.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.206+0000 7f3d1a31d700 1 -- start start 2024-04-25T13:11:17.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d1a31d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 0x7f3d1419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d1a31d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d14103450 0x7f3d1419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d1a31d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d14108a50 0x7f3d1419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d1a31d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14112c20 con 0x7f3d14102a90 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d1a31d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14112aa0 con 0x7f3d14108a50 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d1a31d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14112da0 con 0x7f3d14103450 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d13fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 0x7f3d1419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d13fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 0x7f3d1419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37434/0 (socket says 172.21.15.72:37434) 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d13fff700 1 -- 172.21.15.72:0/563670557 learned_addr learned my addr 172.21.15.72:0/563670557 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d137fe700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d14103450 0x7f3d1419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d13fff700 1 -- 172.21.15.72:0/563670557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d14103450 msgr2=0x7f3d1419b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d188ba700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d14108a50 0x7f3d1419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d13fff700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d14103450 0x7f3d1419b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d13fff700 1 -- 172.21.15.72:0/563670557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d14108a50 msgr2=0x7f3d1419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d13fff700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d14108a50 0x7f3d1419fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d13fff700 1 -- 172.21.15.72:0/563670557 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d141a0360 con 0x7f3d14102a90 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d13fff700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 0x7f3d1419b350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400d950 tx=0x7f3d0400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.207+0000 7f3d117fa700 1 -- 172.21.15.72:0/563670557 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04016070 con 0x7f3d14102a90 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.208+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d141a0650 con 0x7f3d14102a90 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.208+0000 7f3d117fa700 1 -- 172.21.15.72:0/563670557 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d04009df0 con 0x7f3d14102a90 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.208+0000 7f3d117fa700 1 -- 172.21.15.72:0/563670557 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400f3c0 con 0x7f3d14102a90 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.208+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d141a7ed0 con 0x7f3d14102a90 2024-04-25T13:11:17.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.209+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d14069730 con 0x7f3d14102a90 2024-04-25T13:11:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.210+0000 7f3d117fa700 1 -- 172.21.15.72:0/563670557 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3d04019070 con 0x7f3d14102a90 2024-04-25T13:11:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.210+0000 7f3d117fa700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3cfc0a3cd0 0x7f3cfc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.210+0000 7f3d137fe700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3cfc0a3cd0 0x7f3cfc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.210+0000 7f3d117fa700 1 -- 172.21.15.72:0/563670557 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3d040cfff0 con 0x7f3d14102a90 2024-04-25T13:11:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.211+0000 7f3d137fe700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3cfc0a3cd0 0x7f3cfc0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800b2e0 tx=0x7f3d08005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:17.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.215+0000 7f3d117fa700 1 -- 172.21.15.72:0/563670557 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d0409c960 con 0x7f3d14102a90 2024-04-25T13:11:17.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.473+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f3d141a80a0 con 0x7f3d14102a90 2024-04-25T13:11:17.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.478+0000 7f3d117fa700 1 -- 172.21.15.72:0/563670557 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264998 (secure 0 0 0) 0x7f3d0409c260 con 0x7f3d14102a90 2024-04-25T13:11:17.481 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3cfc0a3cd0 msgr2=0x7f3cfc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3cfc0a3cd0 0x7f3cfc0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800b2e0 tx=0x7f3d08005960 comp rx=0 tx=0).stop 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 msgr2=0x7f3d1419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 0x7f3d1419b350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400d950 tx=0x7f3d0400dc60 comp rx=0 tx=0).stop 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 shutdown_connections 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d14108a50 0x7f3d1419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3cfc0a3cd0 0x7f3cfc0a6190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d14102a90 0x7f3d1419b350 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 --2- 172.21.15.72:0/563670557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d14103450 0x7f3d1419b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:17.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 >> 172.21.15.72:0/563670557 conn(0x7f3d14079ba0 msgr2=0x7f3d1410df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:17.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 shutdown_connections 2024-04-25T13:11:17.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:17.482+0000 7f3d1a31d700 1 -- 172.21.15.72:0/563670557 wait complete. 2024-04-25T13:11:18.125 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3348457169},{"type":"v1","addr":"172.21.15.72:6801","nonce":3348457169}]},"active_addr":"172.21.15.72:6801/3348457169","active_change":"2024-04-25T13:04:07.045858+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.72:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":2743588764}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":4004940606}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":3501290813}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":1633858068}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":2476420969}]}]} 2024-04-25T13:11:18.132 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T13:11:18.132 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T13:11:18.132 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:11:18.373 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:18.493 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:18 smithi072 ceph-mon[29199]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-04-25T13:11:18.493 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:18 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/563670557' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:11:18.608 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:18 smithi164 ceph-mon[32418]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-04-25T13:11:18.608 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:18 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/563670557' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:11:18.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:18 smithi193 ceph-mon[32743]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-04-25T13:11:18.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:18 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/563670557' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:11:19.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 -- 172.21.15.72:0/1990789958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806afe0 msgr2=0x7f81a806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:19.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 --2- 172.21.15.72:0/1990789958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806afe0 0x7f81a806b440 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8198009990 tx=0x7f8198009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:19.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 -- 172.21.15.72:0/1990789958 shutdown_connections 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 --2- 172.21.15.72:0/1990789958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a806b980 0x7f81a810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 --2- 172.21.15.72:0/1990789958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a8108a20 0x7f81a8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 --2- 172.21.15.72:0/1990789958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806afe0 0x7f81a806b440 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 -- 172.21.15.72:0/1990789958 >> 172.21.15.72:0/1990789958 conn(0x7f81a80788b0 msgr2=0x7f81a8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 -- 172.21.15.72:0/1990789958 shutdown_connections 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 -- 172.21.15.72:0/1990789958 wait complete. 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 Processor -- start 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.965+0000 7f81ae961700 1 -- start start 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81ae961700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a806afe0 0x7f81a8101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81ae961700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806b980 0x7f81a8101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81ae961700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a8108a20 0x7f81a8102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81ae961700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8112be0 con 0x7f81a806b980 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81ae961700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8112a60 con 0x7f81a8108a20 2024-04-25T13:11:19.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81ae961700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8112d60 con 0x7f81a806afe0 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81a77fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806b980 0x7f81a8101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81a77fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806b980 0x7f81a8101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37442/0 (socket says 172.21.15.72:37442) 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81a77fe700 1 -- 172.21.15.72:0/2942934575 learned_addr learned my addr 172.21.15.72:0/2942934575 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81a7fff700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a806afe0 0x7f81a8101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81acefe700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a8108a20 0x7f81a8102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81a77fe700 1 -- 172.21.15.72:0/2942934575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a806afe0 msgr2=0x7f81a8101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81a77fe700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a806afe0 0x7f81a8101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81a77fe700 1 -- 172.21.15.72:0/2942934575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a8108a20 msgr2=0x7f81a8102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81a77fe700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a8108a20 0x7f81a8102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81a77fe700 1 -- 172.21.15.72:0/2942934575 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a81a7640 con 0x7f81a806b980 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.966+0000 7f81acefe700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a8108a20 0x7f81a8102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.967+0000 7f81a77fe700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806b980 0x7f81a8101f40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8198000c00 tx=0x7f8198009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.967+0000 7f81a57fa700 1 -- 172.21.15.72:0/2942934575 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198027070 con 0x7f81a806b980 2024-04-25T13:11:19.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.967+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a81a78d0 con 0x7f81a806b980 2024-04-25T13:11:19.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.967+0000 7f81a57fa700 1 -- 172.21.15.72:0/2942934575 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8198022470 con 0x7f81a806b980 2024-04-25T13:11:19.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.967+0000 7f81a57fa700 1 -- 172.21.15.72:0/2942934575 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198017650 con 0x7f81a806b980 2024-04-25T13:11:19.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.967+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a81a7d20 con 0x7f81a806b980 2024-04-25T13:11:19.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.968+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a8069730 con 0x7f81a806b980 2024-04-25T13:11:19.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.969+0000 7f81a57fa700 1 -- 172.21.15.72:0/2942934575 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f819802a070 con 0x7f81a806b980 2024-04-25T13:11:19.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.970+0000 7f81a57fa700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81900a3cd0 0x7f81900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:19.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.970+0000 7f81a7fff700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81900a3cd0 0x7f81900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:19.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.970+0000 7f81a57fa700 1 -- 172.21.15.72:0/2942934575 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f81980d1640 con 0x7f81a806b980 2024-04-25T13:11:19.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.970+0000 7f81a7fff700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81900a3cd0 0x7f81900a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f819c007b60 tx=0x7f819c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:19.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:19.973+0000 7f81a57fa700 1 -- 172.21.15.72:0/2942934575 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8198013070 con 0x7f81a806b980 2024-04-25T13:11:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.182+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81a802e140 con 0x7f81a806b980 2024-04-25T13:11:20.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.184+0000 7f81a57fa700 1 -- 172.21.15.72:0/2942934575 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22243 (secure 0 0 0) 0x7f819809e100 con 0x7f81a806b980 2024-04-25T13:11:20.184 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:20.184 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":88,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:11:06.264801+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:11:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81900a3cd0 msgr2=0x7f81900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81900a3cd0 0x7f81900a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f819c007b60 tx=0x7f819c0073d0 comp rx=0 tx=0).stop 2024-04-25T13:11:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806b980 msgr2=0x7f81a8101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806b980 0x7f81a8101f40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8198000c00 tx=0x7f8198009580 comp rx=0 tx=0).stop 2024-04-25T13:11:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 shutdown_connections 2024-04-25T13:11:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a8108a20 0x7f81a8102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a806afe0 0x7f81a8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81900a3cd0 0x7f81900a6190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 --2- 172.21.15.72:0/2942934575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a806b980 0x7f81a8101f40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 >> 172.21.15.72:0/2942934575 conn(0x7f81a80788b0 msgr2=0x7f81a80ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 shutdown_connections 2024-04-25T13:11:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:20.186+0000 7f81ae961700 1 -- 172.21.15.72:0/2942934575 wait complete. 2024-04-25T13:11:20.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:20 smithi193 ceph-mon[32743]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-04-25T13:11:20.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:20 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2942934575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:20.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:20 smithi072 ceph-mon[29199]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-04-25T13:11:20.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:20 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2942934575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:20.807 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T13:11:20.807 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:11:20.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:20 smithi164 ceph-mon[32418]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-04-25T13:11:20.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:20 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2942934575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:21.062 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:22.453 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:22 smithi072 ceph-mon[29199]: pgmap v295: 97 pgs: 97 active+clean; 623 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-25T13:11:22.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.702+0000 7f6b3b87b700 1 -- 172.21.15.72:0/1827216378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 msgr2=0x7f6b34108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:22.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.702+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/1827216378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 0x7f6b34108e30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009990 tx=0x7f6b24009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.702+0000 7f6b3b87b700 1 -- 172.21.15.72:0/1827216378 shutdown_connections 2024-04-25T13:11:22.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.702+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/1827216378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b34103450 0x7f6b3410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.702+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/1827216378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 0x7f6b34108e30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.702+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/1827216378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b34102a90 0x7f6b34102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.702+0000 7f6b3b87b700 1 -- 172.21.15.72:0/1827216378 >> 172.21.15.72:0/1827216378 conn(0x7f6b34079ba0 msgr2=0x7f6b34079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:22.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.703+0000 7f6b3b87b700 1 -- 172.21.15.72:0/1827216378 shutdown_connections 2024-04-25T13:11:22.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.703+0000 7f6b3b87b700 1 -- 172.21.15.72:0/1827216378 wait complete. 2024-04-25T13:11:22.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.703+0000 7f6b3b87b700 1 Processor -- start 2024-04-25T13:11:22.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.703+0000 7f6b3b87b700 1 -- start start 2024-04-25T13:11:22.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b3b87b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b34102a90 0x7f6b34199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:22.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b3b87b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b34103450 0x7f6b34199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:22.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b3b87b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 0x7f6b3419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:22.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b3b87b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34112ce0 con 0x7f6b34108a50 2024-04-25T13:11:22.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b3b87b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34112b60 con 0x7f6b34102a90 2024-04-25T13:11:22.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b3b87b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34112e60 con 0x7f6b34103450 2024-04-25T13:11:22.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39e18700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 0x7f6b3419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39e18700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 0x7f6b3419dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37462/0 (socket says 172.21.15.72:37462) 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39e18700 1 -- 172.21.15.72:0/468760731 learned_addr learned my addr 172.21.15.72:0/468760731 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39617700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b34102a90 0x7f6b34199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b38e16700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b34103450 0x7f6b34199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39e18700 1 -- 172.21.15.72:0/468760731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b34103450 msgr2=0x7f6b34199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39e18700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b34103450 0x7f6b34199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39e18700 1 -- 172.21.15.72:0/468760731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b34102a90 msgr2=0x7f6b34199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39e18700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b34102a90 0x7f6b34199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39e18700 1 -- 172.21.15.72:0/468760731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3419e340 con 0x7f6b34108a50 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b39e18700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 0x7f6b3419dc00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00b970 tx=0x7f6b2c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b2a7fc700 1 -- 172.21.15.72:0/468760731 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c019070 con 0x7f6b34108a50 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3419e630 con 0x7f6b34108a50 2024-04-25T13:11:22.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b2a7fc700 1 -- 172.21.15.72:0/468760731 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b2c00cde0 con 0x7f6b34108a50 2024-04-25T13:11:22.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.704+0000 7f6b2a7fc700 1 -- 172.21.15.72:0/468760731 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c007370 con 0x7f6b34108a50 2024-04-25T13:11:22.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.705+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b341a5eb0 con 0x7f6b34108a50 2024-04-25T13:11:22.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.706+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b34069730 con 0x7f6b34108a50 2024-04-25T13:11:22.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.706+0000 7f6b2a7fc700 1 -- 172.21.15.72:0/468760731 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b2c01c070 con 0x7f6b34108a50 2024-04-25T13:11:22.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.707+0000 7f6b2a7fc700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6b200a3cd0 0x7f6b200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:22.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.707+0000 7f6b39617700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6b200a3cd0 0x7f6b200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:22.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.707+0000 7f6b2a7fc700 1 -- 172.21.15.72:0/468760731 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f6b2c0cf7a0 con 0x7f6b34108a50 2024-04-25T13:11:22.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.708+0000 7f6b39617700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6b200a3cd0 0x7f6b200a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400b5c0 tx=0x7f6b24005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:22.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.712+0000 7f6b2a7fc700 1 -- 172.21.15.72:0/468760731 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b2c09c2e0 con 0x7f6b34108a50 2024-04-25T13:11:22.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:22 smithi193 ceph-mon[32743]: pgmap v295: 97 pgs: 97 active+clean; 623 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-25T13:11:22.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:22 smithi164 ceph-mon[32418]: pgmap v295: 97 pgs: 97 active+clean; 623 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-25T13:11:22.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.909+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b3402df80 con 0x7f6b34108a50 2024-04-25T13:11:22.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.911+0000 7f6b2a7fc700 1 -- 172.21.15.72:0/468760731 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22243 (secure 0 0 0) 0x7f6b2c09bbe0 con 0x7f6b34108a50 2024-04-25T13:11:22.911 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:11:22.915 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":88,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:11:06.264801+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:11:22.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6b200a3cd0 msgr2=0x7f6b200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:22.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6b200a3cd0 0x7f6b200a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400b5c0 tx=0x7f6b24005960 comp rx=0 tx=0).stop 2024-04-25T13:11:22.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 msgr2=0x7f6b3419dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:22.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 0x7f6b3419dc00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00b970 tx=0x7f6b2c00bc80 comp rx=0 tx=0).stop 2024-04-25T13:11:22.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 shutdown_connections 2024-04-25T13:11:22.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6b200a3cd0 0x7f6b200a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b34108a50 0x7f6b3419dc00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b34102a90 0x7f6b34199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 --2- 172.21.15.72:0/468760731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b34103450 0x7f6b34199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:22.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 >> 172.21.15.72:0/468760731 conn(0x7f6b34079ba0 msgr2=0x7f6b34101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:22.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 shutdown_connections 2024-04-25T13:11:22.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:22.913+0000 7f6b3b87b700 1 -- 172.21.15.72:0/468760731 wait complete. 2024-04-25T13:11:23.539 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T13:11:23.539 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T13:11:23.540 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T13:11:23.540 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T13:11:23.540 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T13:11:23.541 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T13:11:23.541 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T13:11:23.542 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T13:11:23.542 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T13:11:23.543 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T13:11:23.543 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T13:11:23.543 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T13:11:23.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:23 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/468760731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:23.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:23 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/468760731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:23.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:23 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/468760731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:11:24.208 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:24.212 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:24.223 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:24.227 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:24.653 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:24 smithi072 ceph-mon[29199]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-25T13:11:24.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:24 smithi193 ceph-mon[32743]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-25T13:11:24.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:24 smithi164 ceph-mon[32418]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-25T13:11:25.374 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:25.374 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:25.375 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:25.375 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:25.376 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:25.376 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:25.388 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:26.232 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:26.629 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:26 smithi072 ceph-mon[29199]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-25T13:11:26.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:26 smithi193 ceph-mon[32743]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-25T13:11:26.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:26 smithi164 ceph-mon[32418]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-25T13:11:28.435 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:28 smithi072 ceph-mon[29199]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T13:11:28.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:28 smithi193 ceph-mon[32743]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T13:11:28.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:28 smithi164 ceph-mon[32418]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T13:11:30.399 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:30 smithi072 ceph-mon[29199]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-25T13:11:30.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:30 smithi193 ceph-mon[32743]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-25T13:11:30.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:30 smithi164 ceph-mon[32418]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-25T13:11:32.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:32 smithi072 ceph-mon[29199]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T13:11:32.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:32 smithi193 ceph-mon[32743]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T13:11:32.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:32 smithi164 ceph-mon[32418]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T13:11:34.581 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:34 smithi072 ceph-mon[29199]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:11:34.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:34 smithi193 ceph-mon[32743]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:11:34.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:34 smithi164 ceph-mon[32418]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:11:35.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 -- 172.21.15.72:0/3096926792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c06bb30 msgr2=0x7f814c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:35.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 --2- 172.21.15.72:0/3096926792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c06bb30 0x7f814c10f900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f814400b0a0 tx=0x7f814400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:11:35.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 -- 172.21.15.72:0/3096926792 shutdown_connections 2024-04-25T13:11:35.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 --2- 172.21.15.72:0/3096926792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c06bb30 0x7f814c10f900 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:35.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 --2- 172.21.15.72:0/3096926792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f814c108a30 0x7f814c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:35.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 --2- 172.21.15.72:0/3096926792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f814c06b190 0x7f814c06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:35.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 -- 172.21.15.72:0/3096926792 >> 172.21.15.72:0/3096926792 conn(0x7f814c078910 msgr2=0x7f814c078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:35.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 -- 172.21.15.72:0/3096926792 shutdown_connections 2024-04-25T13:11:35.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 -- 172.21.15.72:0/3096926792 wait complete. 2024-04-25T13:11:35.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.022+0000 7f8151675700 1 Processor -- start 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f8151675700 1 -- start start 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f8151675700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f814c06b190 0x7f814c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f8151675700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f814c06bb30 0x7f814c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f8151675700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c108a30 0x7f814c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f8151675700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f814c112cb0 con 0x7f814c108a30 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f8151675700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f814c112b30 con 0x7f814c06b190 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f8151675700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f814c112e30 con 0x7f814c06bb30 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c108a30 0x7f814c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c108a30 0x7f814c19db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59386/0 (socket says 172.21.15.72:59386) 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814b7fe700 1 -- 172.21.15.72:0/2088170164 learned_addr learned my addr 172.21.15.72:0/2088170164 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f81427fc700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f814c06bb30 0x7f814c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:35.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814affd700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f814c06b190 0x7f814c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814b7fe700 1 -- 172.21.15.72:0/2088170164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f814c06bb30 msgr2=0x7f814c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814b7fe700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f814c06bb30 0x7f814c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814b7fe700 1 -- 172.21.15.72:0/2088170164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f814c06b190 msgr2=0x7f814c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814b7fe700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f814c06b190 0x7f814c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814b7fe700 1 -- 172.21.15.72:0/2088170164 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c19e270 con 0x7f814c108a30 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814affd700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f814c06b190 0x7f814c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.023+0000 7f814b7fe700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c108a30 0x7f814c19db30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f81440062a0 tx=0x7f8144004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.024+0000 7f8148ff9700 1 -- 172.21.15.72:0/2088170164 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144021070 con 0x7f814c108a30 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.024+0000 7f8148ff9700 1 -- 172.21.15.72:0/2088170164 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8144007d00 con 0x7f814c108a30 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.024+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c19e500 con 0x7f814c108a30 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.024+0000 7f8148ff9700 1 -- 172.21.15.72:0/2088170164 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814401d450 con 0x7f814c108a30 2024-04-25T13:11:35.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.024+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c1a5de0 con 0x7f814c108a30 2024-04-25T13:11:35.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.024+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f814c063190 con 0x7f814c108a30 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.026+0000 7f8148ff9700 1 -- 172.21.15.72:0/2088170164 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8144004020 con 0x7f814c108a30 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.026+0000 7f8148ff9700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81380a3cd0 0x7f81380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.026+0000 7f814affd700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81380a3cd0 0x7f81380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.026+0000 7f8148ff9700 1 -- 172.21.15.72:0/2088170164 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f814400f020 con 0x7f814c108a30 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.027+0000 7f8148ff9700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] conn(0x7f81380ad4b0 0x7f81380af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.027+0000 7f8148ff9700 1 -- 172.21.15.72:0/2088170164 --> [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8144030ea0 con 0x7f81380ad4b0 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.027+0000 7f8148ff9700 1 -- 172.21.15.72:0/2088170164 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f81440d21e0 con 0x7f814c108a30 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.027+0000 7f814affd700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81380a3cd0 0x7f81380a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f813c007900 tx=0x7f813c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.027+0000 7f81427fc700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] conn(0x7f81380ad4b0 0x7f81380af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:35.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.027+0000 7f81427fc700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] conn(0x7f81380ad4b0 0x7f81380af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:35.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.028+0000 7f8148ff9700 1 -- 172.21.15.72:0/2088170164 <== osd.11 v2:172.21.15.193:6824/2763093490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8144030ea0 con 0x7f81380ad4b0 2024-04-25T13:11:35.044 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.044+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 --> [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f814c063590 con 0x7f81380ad4b0 2024-04-25T13:11:35.044 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8148ff9700 1 -- 172.21.15.72:0/2088170164 <== osd.11 v2:172.21.15.193:6824/2763093490 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f814c063590 con 0x7f81380ad4b0 2024-04-25T13:11:35.045 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 >> [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] conn(0x7f81380ad4b0 msgr2=0x7f81380af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:35.045 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] conn(0x7f81380ad4b0 0x7f81380af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:35.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81380a3cd0 msgr2=0x7f81380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:35.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f81380a3cd0 0x7f81380a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f813c007900 tx=0x7f813c008040 comp rx=0 tx=0).stop 2024-04-25T13:11:35.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c108a30 msgr2=0x7f814c19db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:35.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 --2- 172.21.15.72:0/2088170164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c108a30 0x7f814c19db30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f81440062a0 tx=0x7f8144004ed0 comp rx=0 tx=0).stop 2024-04-25T13:11:35.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f814b7fe700 1 -- 172.21.15.72:0/2088170164 reap_dead start 2024-04-25T13:11:35.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 shutdown_connections 2024-04-25T13:11:35.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 >> 172.21.15.72:0/2088170164 conn(0x7f814c078910 msgr2=0x7f814c0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:35.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 shutdown_connections 2024-04-25T13:11:35.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:35.045+0000 7f8151675700 1 -- 172.21.15.72:0/2088170164 wait complete. 2024-04-25T13:11:36.452 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:36 smithi072 ceph-mon[29199]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:11:36.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:36 smithi193 ceph-mon[32743]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:11:36.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:36 smithi164 ceph-mon[32418]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:11:37.249 INFO:teuthology.orchestra.run.smithi072.stdout:356482285587 2024-04-25T13:11:37.250 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T13:11:38.654 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:38 smithi072 ceph-mon[29199]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:11:38.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:38 smithi193 ceph-mon[32743]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:11:38.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:38 smithi164 ceph-mon[32418]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:11:39.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 -- 172.21.15.72:0/2883217905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6314101970 msgr2=0x7f6314101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 --2- 172.21.15.72:0/2883217905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6314101970 0x7f6314101df0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6304009990 tx=0x7f6304009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 -- 172.21.15.72:0/2883217905 shutdown_connections 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 --2- 172.21.15.72:0/2883217905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6314102330 0x7f631410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 --2- 172.21.15.72:0/2883217905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6314107920 0x7f6314107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 --2- 172.21.15.72:0/2883217905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6314101970 0x7f6314101df0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 -- 172.21.15.72:0/2883217905 >> 172.21.15.72:0/2883217905 conn(0x7f6314078910 msgr2=0x7f6314078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 -- 172.21.15.72:0/2883217905 shutdown_connections 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 -- 172.21.15.72:0/2883217905 wait complete. 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.948+0000 7f631b308700 1 Processor -- start 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f631b308700 1 -- start start 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f631b308700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6314101970 0x7f6314197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f631b308700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6314102330 0x7f6314198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f631b308700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6314107920 0x7f631419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f631b308700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6314111b60 con 0x7f6314102330 2024-04-25T13:11:39.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f631b308700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63141119e0 con 0x7f6314101970 2024-04-25T13:11:39.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f631b308700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6314111ce0 con 0x7f6314107920 2024-04-25T13:11:39.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63188a3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6314102330 0x7f6314198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63188a3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6314102330 0x7f6314198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34154/0 (socket says 172.21.15.72:34154) 2024-04-25T13:11:39.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63188a3700 1 -- 172.21.15.72:0/45464654 learned_addr learned my addr 172.21.15.72:0/45464654 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:39.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63190a4700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6314101970 0x7f6314197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63198a5700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6314107920 0x7f631419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63188a3700 1 -- 172.21.15.72:0/45464654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6314107920 msgr2=0x7f631419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63188a3700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6314107920 0x7f631419c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63188a3700 1 -- 172.21.15.72:0/45464654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6314101970 msgr2=0x7f6314197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63188a3700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6314101970 0x7f6314197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63188a3700 1 -- 172.21.15.72:0/45464654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631419cfe0 con 0x7f6314102330 2024-04-25T13:11:39.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63198a5700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6314107920 0x7f631419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f63188a3700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6314102330 0x7f6314198510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6304000c00 tx=0x7f63040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f630a7fc700 1 -- 172.21.15.72:0/45464654 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630402a070 con 0x7f6314102330 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f630a7fc700 1 -- 172.21.15.72:0/45464654 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630401ad70 con 0x7f6314102330 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631419d270 con 0x7f6314102330 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.949+0000 7f630a7fc700 1 -- 172.21.15.72:0/45464654 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400f460 con 0x7f6314102330 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.950+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631406b190 con 0x7f6314102330 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.950+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f631406b740 con 0x7f6314102330 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.951+0000 7f630a7fc700 1 -- 172.21.15.72:0/45464654 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f630402d070 con 0x7f6314102330 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.952+0000 7f630a7fc700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f63000a3cd0 0x7f63000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.952+0000 7f63190a4700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f63000a3cd0 0x7f63000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.952+0000 7f630a7fc700 1 -- 172.21.15.72:0/45464654 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f63040d3bd0 con 0x7f6314102330 2024-04-25T13:11:39.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.952+0000 7f630a7fc700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] conn(0x7f63000ad510 0x7f63000af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.952+0000 7f630a7fc700 1 -- 172.21.15.72:0/45464654 --> [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63000affe0 con 0x7f63000ad510 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.952+0000 7f630a7fc700 1 -- 172.21.15.72:0/45464654 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f63040d40b0 con 0x7f6314102330 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.952+0000 7f63198a5700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] conn(0x7f63000ad510 0x7f63000af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.952+0000 7f63190a4700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f63000a3cd0 0x7f63000a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6310005ea0 tx=0x7f6310005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.953+0000 7f63198a5700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] conn(0x7f63000ad510 0x7f63000af930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.953+0000 7f630a7fc700 1 -- 172.21.15.72:0/45464654 <== osd.3 v2:172.21.15.72:6810/840491303 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63000affe0 con 0x7f63000ad510 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.970+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 --> [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f631406bbc0 con 0x7f63000ad510 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f630a7fc700 1 -- 172.21.15.72:0/45464654 <== osd.3 v2:172.21.15.72:6810/840491303 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f631406bbc0 con 0x7f63000ad510 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] conn(0x7f63000ad510 msgr2=0x7f63000af930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] conn(0x7f63000ad510 0x7f63000af930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f63000a3cd0 msgr2=0x7f63000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f63000a3cd0 0x7f63000a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6310005ea0 tx=0x7f6310005e30 comp rx=0 tx=0).stop 2024-04-25T13:11:39.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6314102330 msgr2=0x7f6314198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 --2- 172.21.15.72:0/45464654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6314102330 0x7f6314198510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6304000c00 tx=0x7f63040047b0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f63198a5700 1 -- 172.21.15.72:0/45464654 reap_dead start 2024-04-25T13:11:39.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 shutdown_connections 2024-04-25T13:11:39.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 >> 172.21.15.72:0/45464654 conn(0x7f6314078910 msgr2=0x7f63140ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:39.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 shutdown_connections 2024-04-25T13:11:39.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:38.971+0000 7f631b308700 1 -- 172.21.15.72:0/45464654 wait complete. 2024-04-25T13:11:39.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.907+0000 7f906cf08700 1 -- 172.21.15.72:0/1122910071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9068101970 msgr2=0x7f9068101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.907+0000 7f906cf08700 1 --2- 172.21.15.72:0/1122910071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9068101970 0x7f9068101df0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f905c009990 tx=0x7f905c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.907+0000 7f906cf08700 1 -- 172.21.15.72:0/1122910071 shutdown_connections 2024-04-25T13:11:39.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.907+0000 7f906cf08700 1 --2- 172.21.15.72:0/1122910071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9068102330 0x7f906810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.907+0000 7f906cf08700 1 --2- 172.21.15.72:0/1122910071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9068107920 0x7f9068107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.907+0000 7f906cf08700 1 --2- 172.21.15.72:0/1122910071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9068101970 0x7f9068101df0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.908+0000 7f906cf08700 1 -- 172.21.15.72:0/1122910071 >> 172.21.15.72:0/1122910071 conn(0x7f9068078910 msgr2=0x7f9068078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:39.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.908+0000 7f906cf08700 1 -- 172.21.15.72:0/1122910071 shutdown_connections 2024-04-25T13:11:39.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.908+0000 7f906cf08700 1 -- 172.21.15.72:0/1122910071 wait complete. 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.908+0000 7f906cf08700 1 Processor -- start 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.908+0000 7f906cf08700 1 -- start start 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.908+0000 7f906cf08700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9068101970 0x7f9068198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.908+0000 7f906cf08700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9068102330 0x7f9068198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f906cf08700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9068107920 0x7f906819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f906cf08700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9068111b30 con 0x7f9068107920 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f906cf08700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90681119b0 con 0x7f9068102330 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f906cf08700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9068111cb0 con 0x7f9068101970 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9066d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9068107920 0x7f906819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9066d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9068107920 0x7f906819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34200/0 (socket says 172.21.15.72:34200) 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9066d9d700 1 -- 172.21.15.72:0/3660955790 learned_addr learned my addr 172.21.15.72:0/3660955790 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:39.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f906659c700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9068101970 0x7f9068198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9065d9b700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9068102330 0x7f9068198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9066d9d700 1 -- 172.21.15.72:0/3660955790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9068101970 msgr2=0x7f9068198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9066d9d700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9068101970 0x7f9068198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9066d9d700 1 -- 172.21.15.72:0/3660955790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9068102330 msgr2=0x7f9068198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9066d9d700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9068102330 0x7f9068198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9066d9d700 1 -- 172.21.15.72:0/3660955790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906819d010 con 0x7f9068107920 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9065d9b700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9068102330 0x7f9068198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f9066d9d700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9068107920 0x7f906819c8d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9060007ee0 tx=0x7f906000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f90577fe700 1 -- 172.21.15.72:0/3660955790 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906000dbd0 con 0x7f9068107920 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f90577fe700 1 -- 172.21.15.72:0/3660955790 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906000dd70 con 0x7f9068107920 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f90577fe700 1 -- 172.21.15.72:0/3660955790 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060005990 con 0x7f9068107920 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906819d300 con 0x7f9068107920 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.909+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90681a4b80 con 0x7f9068107920 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.910+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9068063190 con 0x7f9068107920 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.912+0000 7f90577fe700 1 -- 172.21.15.72:0/3660955790 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f906001a020 con 0x7f9068107920 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.912+0000 7f90577fe700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f90500a3cd0 0x7f90500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.912+0000 7f906659c700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f90500a3cd0 0x7f90500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.912+0000 7f90577fe700 1 -- 172.21.15.72:0/3660955790 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f90600d01f0 con 0x7f9068107920 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.912+0000 7f90577fe700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] conn(0x7f90500ad4b0 0x7f90500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.912+0000 7f90577fe700 1 -- 172.21.15.72:0/3660955790 --> [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90500aff80 con 0x7f90500ad4b0 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.912+0000 7f90577fe700 1 -- 172.21.15.72:0/3660955790 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f90600d06d0 con 0x7f9068107920 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.913+0000 7f906659c700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f90500a3cd0 0x7f90500a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9058005950 tx=0x7f905800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.913+0000 7f9065d9b700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] conn(0x7f90500ad4b0 0x7f90500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.913+0000 7f9065d9b700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] conn(0x7f90500ad4b0 0x7f90500af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.914+0000 7f90577fe700 1 -- 172.21.15.72:0/3660955790 <== osd.8 v2:172.21.15.193:6816/2354460844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90500aff80 con 0x7f90500ad4b0 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 --> [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9068063610 con 0x7f90500ad4b0 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f90577fe700 1 -- 172.21.15.72:0/3660955790 <== osd.8 v2:172.21.15.193:6816/2354460844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9068063610 con 0x7f90500ad4b0 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 >> [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] conn(0x7f90500ad4b0 msgr2=0x7f90500af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f906cf08700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] conn(0x7f90500ad4b0 0x7f90500af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f90500a3cd0 msgr2=0x7f90500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f906cf08700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f90500a3cd0 0x7f90500a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9058005950 tx=0x7f905800a300 comp rx=0 tx=0).stop 2024-04-25T13:11:39.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9068107920 msgr2=0x7f906819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f906cf08700 1 --2- 172.21.15.72:0/3660955790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9068107920 0x7f906819c8d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9060007ee0 tx=0x7f906000d3b0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f9066d9d700 1 -- 172.21.15.72:0/3660955790 reap_dead start 2024-04-25T13:11:39.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 shutdown_connections 2024-04-25T13:11:39.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.930+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 >> 172.21.15.72:0/3660955790 conn(0x7f9068078910 msgr2=0x7f90680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:39.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.931+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 shutdown_connections 2024-04-25T13:11:39.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.931+0000 7f906cf08700 1 -- 172.21.15.72:0/3660955790 wait complete. 2024-04-25T13:11:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.803+0000 7fcbd3981700 1 -- 172.21.15.72:0/2456772339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbcc107980 msgr2=0x7fcbcc107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.803+0000 7fcbd3981700 1 --2- 172.21.15.72:0/2456772339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbcc107980 0x7fcbcc107d60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009990 tx=0x7fcbbc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.803+0000 7fcbd3981700 1 -- 172.21.15.72:0/2456772339 shutdown_connections 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.803+0000 7fcbd3981700 1 --2- 172.21.15.72:0/2456772339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbcc1022f0 0x7fcbcc10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.803+0000 7fcbd3981700 1 --2- 172.21.15.72:0/2456772339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbcc107980 0x7fcbcc107d60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.803+0000 7fcbd3981700 1 --2- 172.21.15.72:0/2456772339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc101930 0x7fcbcc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.803+0000 7fcbd3981700 1 -- 172.21.15.72:0/2456772339 >> 172.21.15.72:0/2456772339 conn(0x7fcbcc0788b0 msgr2=0x7fcbcc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.803+0000 7fcbd3981700 1 -- 172.21.15.72:0/2456772339 shutdown_connections 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.803+0000 7fcbd3981700 1 -- 172.21.15.72:0/2456772339 wait complete. 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd3981700 1 Processor -- start 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd3981700 1 -- start start 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd3981700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc101930 0x7fcbcc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd3981700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbcc1022f0 0x7fcbcc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd0f1c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbcc1022f0 0x7fcbcc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd3981700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbcc107980 0x7fcbcc19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd0f1c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbcc1022f0 0x7fcbcc1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34182/0 (socket says 172.21.15.72:34182) 2024-04-25T13:11:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd3981700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc111b50 con 0x7fcbcc1022f0 2024-04-25T13:11:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd0f1c700 1 -- 172.21.15.72:0/2654952212 learned_addr learned my addr 172.21.15.72:0/2654952212 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1119d0 con 0x7fcbcc101930 2024-04-25T13:11:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.804+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc111cd0 con 0x7fcbcc107980 2024-04-25T13:11:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbd0f1c700 1 -- 172.21.15.72:0/2654952212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbcc107980 msgr2=0x7fcbcc19c950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:11:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbd0f1c700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbcc107980 0x7fcbcc19c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbd0f1c700 1 -- 172.21.15.72:0/2654952212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc101930 msgr2=0x7fcbcc198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbd0f1c700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc101930 0x7fcbcc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbd0f1c700 1 -- 172.21.15.72:0/2654952212 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc19d090 con 0x7fcbcc1022f0 2024-04-25T13:11:40.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbd0f1c700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbcc1022f0 0x7fcbcc1985c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800b530 tx=0x7fcbc800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:40.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbc27fc700 1 -- 172.21.15.72:0/2654952212 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8015020 con 0x7fcbcc1022f0 2024-04-25T13:11:40.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbcc19d380 con 0x7fcbcc1022f0 2024-04-25T13:11:40.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbcc06afb0 con 0x7fcbcc1022f0 2024-04-25T13:11:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbc27fc700 1 -- 172.21.15.72:0/2654952212 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc8011ce0 con 0x7fcbcc1022f0 2024-04-25T13:11:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.805+0000 7fcbc27fc700 1 -- 172.21.15.72:0/2654952212 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8010d80 con 0x7fcbcc1022f0 2024-04-25T13:11:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.806+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcbb0001060 con 0x7fcbcc1022f0 2024-04-25T13:11:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.808+0000 7fcbc27fc700 1 -- 172.21.15.72:0/2654952212 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcbc8018020 con 0x7fcbcc1022f0 2024-04-25T13:11:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.808+0000 7fcbc27fc700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcbb80a3d50 0x7fcbb80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.809+0000 7fcbd171d700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcbb80a3d50 0x7fcbb80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.809+0000 7fcbc27fc700 1 -- 172.21.15.72:0/2654952212 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fcbc80d0080 con 0x7fcbcc1022f0 2024-04-25T13:11:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.809+0000 7fcbc27fc700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] conn(0x7fcbb80ad530 0x7fcbb80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.809+0000 7fcbc27fc700 1 -- 172.21.15.72:0/2654952212 --> [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcbc8016ec0 con 0x7fcbb80ad530 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.809+0000 7fcbc27fc700 1 -- 172.21.15.72:0/2654952212 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fcbc80d04e0 con 0x7fcbcc1022f0 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.809+0000 7fcbd171d700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcbb80a3d50 0x7fcbb80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0098e0 tx=0x7fcbbc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.809+0000 7fcbd1f1e700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] conn(0x7fcbb80ad530 0x7fcbb80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.809+0000 7fcbd1f1e700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] conn(0x7fcbb80ad530 0x7fcbb80af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.810+0000 7fcbc27fc700 1 -- 172.21.15.72:0/2654952212 <== osd.9 v2:172.21.15.72:6826/4174455071 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbc8016ec0 con 0x7fcbb80ad530 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.826+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 --> [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcbb0002e90 con 0x7fcbb80ad530 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbc27fc700 1 -- 172.21.15.72:0/2654952212 <== osd.9 v2:172.21.15.72:6826/4174455071 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcbb0002e90 con 0x7fcbb80ad530 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] conn(0x7fcbb80ad530 msgr2=0x7fcbb80af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] conn(0x7fcbb80ad530 0x7fcbb80af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcbb80a3d50 msgr2=0x7fcbb80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fcbb80a3d50 0x7fcbb80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0098e0 tx=0x7fcbbc005fb0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbcc1022f0 msgr2=0x7fcbcc1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 --2- 172.21.15.72:0/2654952212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbcc1022f0 0x7fcbcc1985c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800b530 tx=0x7fcbc800b8f0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd1f1e700 1 -- 172.21.15.72:0/2654952212 reap_dead start 2024-04-25T13:11:40.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 shutdown_connections 2024-04-25T13:11:40.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 >> 172.21.15.72:0/2654952212 conn(0x7fcbcc0788b0 msgr2=0x7fcbcc1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:40.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 shutdown_connections 2024-04-25T13:11:40.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:39.827+0000 7fcbd3981700 1 -- 172.21.15.72:0/2654952212 wait complete. 2024-04-25T13:11:40.698 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:40 smithi072 ceph-mon[29199]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:11:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:40 smithi193 ceph-mon[32743]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:11:40.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:40 smithi164 ceph-mon[32418]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:11:41.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.825+0000 7f0e935bb700 1 -- 172.21.15.72:0/3632134939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e8c102b00 msgr2=0x7f0e8c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.825+0000 7f0e935bb700 1 --2- 172.21.15.72:0/3632134939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e8c102b00 0x7f0e8c102f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009990 tx=0x7f0e7c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.825+0000 7f0e935bb700 1 -- 172.21.15.72:0/3632134939 shutdown_connections 2024-04-25T13:11:41.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.825+0000 7f0e935bb700 1 --2- 172.21.15.72:0/3632134939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c1034c0 0x7f0e8c10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.825+0000 7f0e935bb700 1 --2- 172.21.15.72:0/3632134939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e8c108a10 0x7f0e8c108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.825+0000 7f0e935bb700 1 --2- 172.21.15.72:0/3632134939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e8c102b00 0x7f0e8c102f80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.825+0000 7f0e935bb700 1 -- 172.21.15.72:0/3632134939 >> 172.21.15.72:0/3632134939 conn(0x7f0e8c079c00 msgr2=0x7f0e8c07a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.825+0000 7f0e935bb700 1 -- 172.21.15.72:0/3632134939 shutdown_connections 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.825+0000 7f0e935bb700 1 -- 172.21.15.72:0/3632134939 wait complete. 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e935bb700 1 Processor -- start 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e935bb700 1 -- start start 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e935bb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e8c102b00 0x7f0e8c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e935bb700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e8c1034c0 0x7f0e8c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e935bb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108a10 0x7f0e8c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e935bb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c112c10 con 0x7f0e8c102b00 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e935bb700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c112a90 con 0x7f0e8c1034c0 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e935bb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c112d90 con 0x7f0e8c108a10 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91357700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e8c102b00 0x7f0e8c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91357700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e8c102b00 0x7f0e8c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34240/0 (socket says 172.21.15.72:34240) 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91357700 1 -- 172.21.15.72:0/3804629117 learned_addr learned my addr 172.21.15.72:0/3804629117 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91b58700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108a10 0x7f0e8c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91357700 1 -- 172.21.15.72:0/3804629117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108a10 msgr2=0x7f0e8c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91357700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108a10 0x7f0e8c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91357700 1 -- 172.21.15.72:0/3804629117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e8c1034c0 msgr2=0x7f0e8c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91357700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e8c1034c0 0x7f0e8c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91357700 1 -- 172.21.15.72:0/3804629117 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c1a7600 con 0x7f0e8c102b00 2024-04-25T13:11:41.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e91357700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e8c102b00 0x7f0e8c074d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800d980 tx=0x7f0e8800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e827fc700 1 -- 172.21.15.72:0/3804629117 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88016070 con 0x7f0e8c102b00 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c1a78f0 con 0x7f0e8c102b00 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e827fc700 1 -- 172.21.15.72:0/3804629117 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e880044d0 con 0x7f0e8c102b00 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.826+0000 7f0e827fc700 1 -- 172.21.15.72:0/3804629117 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88004d00 con 0x7f0e8c102b00 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.827+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c1a7e80 con 0x7f0e8c102b00 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.827+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e8c063110 con 0x7f0e8c102b00 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.829+0000 7f0e827fc700 1 -- 172.21.15.72:0/3804629117 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e88004020 con 0x7f0e8c102b00 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.829+0000 7f0e827fc700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0e780a3cd0 0x7f0e780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.829+0000 7f0e90b56700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0e780a3cd0 0x7f0e780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.829+0000 7f0e827fc700 1 -- 172.21.15.72:0/3804629117 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0e880d0900 con 0x7f0e8c102b00 2024-04-25T13:11:41.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.829+0000 7f0e827fc700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] conn(0x7f0e780ad4b0 0x7f0e780af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.829+0000 7f0e827fc700 1 -- 172.21.15.72:0/3804629117 --> [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e88015ec0 con 0x7f0e780ad4b0 2024-04-25T13:11:41.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.829+0000 7f0e827fc700 1 -- 172.21.15.72:0/3804629117 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0e880d0de0 con 0x7f0e8c102b00 2024-04-25T13:11:41.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.830+0000 7f0e91b58700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] conn(0x7f0e780ad4b0 0x7f0e780af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:41.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.830+0000 7f0e90b56700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0e780a3cd0 0x7f0e780a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0098e0 tx=0x7f0e7c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:41.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.830+0000 7f0e91b58700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] conn(0x7f0e780ad4b0 0x7f0e780af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:41.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.831+0000 7f0e827fc700 1 -- 172.21.15.72:0/3804629117 <== osd.6 v2:172.21.15.72:6818/3792057765 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e88015ec0 con 0x7f0e780ad4b0 2024-04-25T13:11:41.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 --> [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e8c063510 con 0x7f0e780ad4b0 2024-04-25T13:11:41.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e827fc700 1 -- 172.21.15.72:0/3804629117 <== osd.6 v2:172.21.15.72:6818/3792057765 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0e8c063510 con 0x7f0e780ad4b0 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] conn(0x7f0e780ad4b0 msgr2=0x7f0e780af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] conn(0x7f0e780ad4b0 0x7f0e780af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0e780a3cd0 msgr2=0x7f0e780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0e780a3cd0 0x7f0e780a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0098e0 tx=0x7f0e7c005fb0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e8c102b00 msgr2=0x7f0e8c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 --2- 172.21.15.72:0/3804629117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e8c102b00 0x7f0e8c074d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800d980 tx=0x7f0e8800dd40 comp rx=0 tx=0).stop 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e91b58700 1 -- 172.21.15.72:0/3804629117 reap_dead start 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 shutdown_connections 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 >> 172.21.15.72:0/3804629117 conn(0x7f0e8c079c00 msgr2=0x7f0e8c100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.847+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 shutdown_connections 2024-04-25T13:11:41.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.848+0000 7f0e935bb700 1 -- 172.21.15.72:0/3804629117 wait complete. 2024-04-25T13:11:41.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.719+0000 7f003b9cf700 1 -- 172.21.15.72:0/1586282250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0034076a20 msgr2=0x7f0034074c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.719+0000 7f003b9cf700 1 --2- 172.21.15.72:0/1586282250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0034076a20 0x7f0034074c70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0030009990 tx=0x7f0030009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.719+0000 7f003b9cf700 1 -- 172.21.15.72:0/1586282250 shutdown_connections 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.719+0000 7f003b9cf700 1 --2- 172.21.15.72:0/1586282250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0034113740 0x7f0034115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.719+0000 7f003b9cf700 1 --2- 172.21.15.72:0/1586282250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0034076a20 0x7f0034074c70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.719+0000 7f003b9cf700 1 --2- 172.21.15.72:0/1586282250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00340751b0 0x7f0034075630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.719+0000 7f003b9cf700 1 -- 172.21.15.72:0/1586282250 >> 172.21.15.72:0/1586282250 conn(0x7f0034079c00 msgr2=0x7f003407a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.719+0000 7f003b9cf700 1 -- 172.21.15.72:0/1586282250 shutdown_connections 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.719+0000 7f003b9cf700 1 -- 172.21.15.72:0/1586282250 wait complete. 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003b9cf700 1 Processor -- start 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003b9cf700 1 -- start start 2024-04-25T13:11:41.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003b9cf700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00340751b0 0x7f00341a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003b9cf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0034076a20 0x7f00341a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003b9cf700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0034113740 0x7f003419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003b9cf700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f00341192d0 con 0x7f00340751b0 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003b9cf700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0034119150 con 0x7f0034113740 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003b9cf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0034119450 con 0x7f0034076a20 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003976b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00340751b0 0x7f00341a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003976b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00340751b0 0x7f00341a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34216/0 (socket says 172.21.15.72:34216) 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003976b700 1 -- 172.21.15.72:0/101917493 learned_addr learned my addr 172.21.15.72:0/101917493 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f0038f6a700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0034076a20 0x7f00341a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f0039f6c700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0034113740 0x7f003419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003976b700 1 -- 172.21.15.72:0/101917493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0034076a20 msgr2=0x7f00341a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.720+0000 7f003976b700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0034076a20 0x7f00341a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f003976b700 1 -- 172.21.15.72:0/101917493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0034113740 msgr2=0x7f003419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f003976b700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0034113740 0x7f003419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f003976b700 1 -- 172.21.15.72:0/101917493 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003419ff70 con 0x7f00340751b0 2024-04-25T13:11:41.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f0039f6c700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0034113740 0x7f003419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:41.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f003976b700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00340751b0 0x7f00341a55c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0030000c00 tx=0x7f00300047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:41.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f002a7fc700 1 -- 172.21.15.72:0/101917493 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030027070 con 0x7f00340751b0 2024-04-25T13:11:41.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00341a0200 con 0x7f00340751b0 2024-04-25T13:11:41.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341ac540 con 0x7f00340751b0 2024-04-25T13:11:41.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f002a7fc700 1 -- 172.21.15.72:0/101917493 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0030003fa0 con 0x7f00340751b0 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.721+0000 7f002a7fc700 1 -- 172.21.15.72:0/101917493 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030022dd0 con 0x7f00340751b0 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.722+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0034112960 con 0x7f00340751b0 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.724+0000 7f002a7fc700 1 -- 172.21.15.72:0/101917493 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f003002a070 con 0x7f00340751b0 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.725+0000 7f002a7fc700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00200a3d50 0x7f00200a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.725+0000 7f0038f6a700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00200a3d50 0x7f00200a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.725+0000 7f002a7fc700 1 -- 172.21.15.72:0/101917493 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f00300d1830 con 0x7f00340751b0 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.725+0000 7f002a7fc700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] conn(0x7f00200ad530 0x7f00200af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.725+0000 7f002a7fc700 1 -- 172.21.15.72:0/101917493 --> [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f003002ee50 con 0x7f00200ad530 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.725+0000 7f002a7fc700 1 -- 172.21.15.72:0/101917493 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f00300d1d10 con 0x7f00340751b0 2024-04-25T13:11:41.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.725+0000 7f0039f6c700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] conn(0x7f00200ad530 0x7f00200af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.726+0000 7f0038f6a700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00200a3d50 0x7f00200a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0024006fd0 tx=0x7f0024008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.726+0000 7f0039f6c700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] conn(0x7f00200ad530 0x7f00200af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.727+0000 7f002a7fc700 1 -- 172.21.15.72:0/101917493 <== osd.5 v2:172.21.15.193:6808/2502247683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f003002ee50 con 0x7f00200ad530 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.743+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 --> [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f00340762d0 con 0x7f00200ad530 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.743+0000 7f002a7fc700 1 -- 172.21.15.72:0/101917493 <== osd.5 v2:172.21.15.193:6808/2502247683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f00340762d0 con 0x7f00200ad530 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 >> [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] conn(0x7f00200ad530 msgr2=0x7f00200af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] conn(0x7f00200ad530 0x7f00200af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00200a3d50 msgr2=0x7f00200a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f00200a3d50 0x7f00200a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0024006fd0 tx=0x7f0024008040 comp rx=0 tx=0).stop 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00340751b0 msgr2=0x7f00341a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 --2- 172.21.15.72:0/101917493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00340751b0 0x7f00341a55c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0030000c00 tx=0x7f00300047b0 comp rx=0 tx=0).stop 2024-04-25T13:11:41.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f0039f6c700 1 -- 172.21.15.72:0/101917493 reap_dead start 2024-04-25T13:11:41.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 shutdown_connections 2024-04-25T13:11:41.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 >> 172.21.15.72:0/101917493 conn(0x7f0034079c00 msgr2=0x7f0034111fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:41.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 shutdown_connections 2024-04-25T13:11:41.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:41.744+0000 7f003b9cf700 1 -- 172.21.15.72:0/101917493 wait complete. 2024-04-25T13:11:42.712 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:42 smithi072 ceph-mon[29199]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:11:42.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:42 smithi193 ceph-mon[32743]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:11:42.740 INFO:teuthology.orchestra.run.smithi072.stdout:107374182460 2024-04-25T13:11:42.741 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T13:11:42.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:42 smithi164 ceph-mon[32418]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:11:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:261993005091 2024-04-25T13:11:43.620 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T13:11:44.147 INFO:teuthology.orchestra.run.smithi072.stdout:296352743453 2024-04-25T13:11:44.148 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T13:11:44.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.907+0000 7f83b7611700 1 -- 172.21.15.72:0/3649332113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b0108a20 msgr2=0x7f83b0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.907+0000 7f83b7611700 1 --2- 172.21.15.72:0/3649332113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b0108a20 0x7f83b0108e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009990 tx=0x7f83ac009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.907+0000 7f83b7611700 1 -- 172.21.15.72:0/3649332113 shutdown_connections 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.907+0000 7f83b7611700 1 --2- 172.21.15.72:0/3649332113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83b006b980 0x7f83b010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.907+0000 7f83b7611700 1 --2- 172.21.15.72:0/3649332113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b0108a20 0x7f83b0108e00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.907+0000 7f83b7611700 1 --2- 172.21.15.72:0/3649332113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b006afe0 0x7f83b006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.907+0000 7f83b7611700 1 -- 172.21.15.72:0/3649332113 >> 172.21.15.72:0/3649332113 conn(0x7f83b00788b0 msgr2=0x7f83b0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.907+0000 7f83b7611700 1 -- 172.21.15.72:0/3649332113 shutdown_connections 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.907+0000 7f83b7611700 1 -- 172.21.15.72:0/3649332113 wait complete. 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b7611700 1 Processor -- start 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b7611700 1 -- start start 2024-04-25T13:11:44.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b7611700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b006afe0 0x7f83b0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b7611700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83b006b980 0x7f83b01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b7611700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b0108a20 0x7f83b019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b7611700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f83b0112ca0 con 0x7f83b006afe0 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b7611700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83b0112b20 con 0x7f83b006b980 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b7611700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83b0112e20 con 0x7f83b0108a20 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b53ad700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b006afe0 0x7f83b0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b53ad700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b006afe0 0x7f83b0199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34262/0 (socket says 172.21.15.72:34262) 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.908+0000 7f83b53ad700 1 -- 172.21.15.72:0/1089002387 learned_addr learned my addr 172.21.15.72:0/1089002387 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83b53ad700 1 -- 172.21.15.72:0/1089002387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b0108a20 msgr2=0x7f83b019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83b53ad700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b0108a20 0x7f83b019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83b53ad700 1 -- 172.21.15.72:0/1089002387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83b006b980 msgr2=0x7f83b01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83b4bac700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83b006b980 0x7f83b01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83b53ad700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83b006b980 0x7f83b01997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83b53ad700 1 -- 172.21.15.72:0/1089002387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b019e290 con 0x7f83b006afe0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83b53ad700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b006afe0 0x7f83b0199280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000c00 tx=0x7f83ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83a67fc700 1 -- 172.21.15.72:0/1089002387 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac027070 con 0x7f83b006afe0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b019e520 con 0x7f83b006afe0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83a67fc700 1 -- 172.21.15.72:0/1089002387 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ac0041d0 con 0x7f83b006afe0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83a67fc700 1 -- 172.21.15.72:0/1089002387 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00f460 con 0x7f83b006afe0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.909+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b01a5e00 con 0x7f83b006afe0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.910+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83b00631d0 con 0x7f83b006afe0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.911+0000 7f83a67fc700 1 -- 172.21.15.72:0/1089002387 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f83ac02a070 con 0x7f83b006afe0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.911+0000 7f83a67fc700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f839c0a3cd0 0x7f839c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.912+0000 7f83b4bac700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f839c0a3cd0 0x7f839c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:44.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.912+0000 7f83a67fc700 1 -- 172.21.15.72:0/1089002387 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f83ac0d1ab0 con 0x7f83b006afe0 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.912+0000 7f83a67fc700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] conn(0x7f839c0ad4b0 0x7f839c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.912+0000 7f83a67fc700 1 -- 172.21.15.72:0/1089002387 --> [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83ac0043d0 con 0x7f839c0ad4b0 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.912+0000 7f83a67fc700 1 -- 172.21.15.72:0/1089002387 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f83ac0d1f90 con 0x7f83b006afe0 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.912+0000 7f83b4bac700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f839c0a3cd0 0x7f839c0a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f83a0006fd0 tx=0x7f83a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.912+0000 7f83b5bae700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] conn(0x7f839c0ad4b0 0x7f839c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.912+0000 7f83b5bae700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] conn(0x7f839c0ad4b0 0x7f839c0af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.913+0000 7f83a67fc700 1 -- 172.21.15.72:0/1089002387 <== osd.2 v2:172.21.15.193:6800/1367843850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83ac0043d0 con 0x7f839c0ad4b0 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.929+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 --> [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83b0063650 con 0x7f839c0ad4b0 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83a67fc700 1 -- 172.21.15.72:0/1089002387 <== osd.2 v2:172.21.15.193:6800/1367843850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83b0063650 con 0x7f839c0ad4b0 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 >> [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] conn(0x7f839c0ad4b0 msgr2=0x7f839c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] conn(0x7f839c0ad4b0 0x7f839c0af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f839c0a3cd0 msgr2=0x7f839c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f839c0a3cd0 0x7f839c0a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f83a0006fd0 tx=0x7f83a0008040 comp rx=0 tx=0).stop 2024-04-25T13:11:44.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b006afe0 msgr2=0x7f83b0199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 --2- 172.21.15.72:0/1089002387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b006afe0 0x7f83b0199280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000c00 tx=0x7f83ac0047b0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b5bae700 1 -- 172.21.15.72:0/1089002387 reap_dead start 2024-04-25T13:11:44.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 shutdown_connections 2024-04-25T13:11:44.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 >> 172.21.15.72:0/1089002387 conn(0x7f83b00788b0 msgr2=0x7f83b00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:44.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 shutdown_connections 2024-04-25T13:11:44.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:43.930+0000 7f83b7611700 1 -- 172.21.15.72:0/1089002387 wait complete. 2024-04-25T13:11:44.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:44 smithi072 ceph-mon[29199]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:11:44.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 -- 172.21.15.72:0/767431306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7688107990 msgr2=0x7f7688107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 --2- 172.21.15.72:0/767431306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7688107990 0x7f7688107d70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7684009990 tx=0x7f7684009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 -- 172.21.15.72:0/767431306 shutdown_connections 2024-04-25T13:11:44.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 --2- 172.21.15.72:0/767431306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7688102300 0x7f768810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 --2- 172.21.15.72:0/767431306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7688107990 0x7f7688107d70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 --2- 172.21.15.72:0/767431306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7688101940 0x7f7688101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 -- 172.21.15.72:0/767431306 >> 172.21.15.72:0/767431306 conn(0x7f76880788f0 msgr2=0x7f7688078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:44.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 -- 172.21.15.72:0/767431306 shutdown_connections 2024-04-25T13:11:44.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 -- 172.21.15.72:0/767431306 wait complete. 2024-04-25T13:11:44.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 Processor -- start 2024-04-25T13:11:44.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.633+0000 7f768f7d8700 1 -- start start 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768f7d8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7688101940 0x7f7688199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768f7d8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7688102300 0x7f76881996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768f7d8700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7688107990 0x7f768819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768f7d8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7688111b00 con 0x7f7688101940 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768f7d8700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7688111980 con 0x7f7688107990 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768f7d8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7688111c80 con 0x7f7688102300 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768d574700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7688101940 0x7f7688199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768d574700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7688101940 0x7f7688199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34292/0 (socket says 172.21.15.72:34292) 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768d574700 1 -- 172.21.15.72:0/1453055978 learned_addr learned my addr 172.21.15.72:0/1453055978 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:44.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768cd73700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7688102300 0x7f76881996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768dd75700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7688107990 0x7f768819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768d574700 1 -- 172.21.15.72:0/1453055978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7688102300 msgr2=0x7f76881996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768d574700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7688102300 0x7f76881996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768d574700 1 -- 172.21.15.72:0/1453055978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7688107990 msgr2=0x7f768819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768d574700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7688107990 0x7f768819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768d574700 1 -- 172.21.15.72:0/1453055978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768819e140 con 0x7f7688101940 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768dd75700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7688107990 0x7f768819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768d574700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7688101940 0x7f7688199180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7684000c00 tx=0x7f76840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f767e7fc700 1 -- 172.21.15.72:0/1453055978 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684027070 con 0x7f7688101940 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768819e3d0 con 0x7f7688101940 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f767e7fc700 1 -- 172.21.15.72:0/1453055978 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7684003f20 con 0x7f7688101940 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f767e7fc700 1 -- 172.21.15.72:0/1453055978 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684022cf0 con 0x7f7688101940 2024-04-25T13:11:44.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.634+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76881a5cb0 con 0x7f7688101940 2024-04-25T13:11:44.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.635+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7688106990 con 0x7f7688101940 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.637+0000 7f767e7fc700 1 -- 172.21.15.72:0/1453055978 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f768402a070 con 0x7f7688101940 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.637+0000 7f767e7fc700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f76740a3cd0 0x7f76740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.637+0000 7f768cd73700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f76740a3cd0 0x7f76740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.637+0000 7f767e7fc700 1 -- 172.21.15.72:0/1453055978 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f76840d17f0 con 0x7f7688101940 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.637+0000 7f767e7fc700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] conn(0x7f76740ad4b0 0x7f76740af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.637+0000 7f767e7fc700 1 -- 172.21.15.72:0/1453055978 --> [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76840095d0 con 0x7f76740ad4b0 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.637+0000 7f767e7fc700 1 -- 172.21.15.72:0/1453055978 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f76840d1cd0 con 0x7f7688101940 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.638+0000 7f768cd73700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f76740a3cd0 0x7f76740a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7688078f40 tx=0x7f767800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.638+0000 7f768dd75700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] conn(0x7f76740ad4b0 0x7f76740af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:44.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.638+0000 7f768dd75700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] conn(0x7f76740ad4b0 0x7f76740af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:44.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.638+0000 7f767e7fc700 1 -- 172.21.15.72:0/1453055978 <== osd.0 v2:172.21.15.72:6802/2197040660 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76840095d0 con 0x7f76740ad4b0 2024-04-25T13:11:44.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 --> [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7688103060 con 0x7f76740ad4b0 2024-04-25T13:11:44.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f767e7fc700 1 -- 172.21.15.72:0/1453055978 <== osd.0 v2:172.21.15.72:6802/2197040660 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7688103060 con 0x7f76740ad4b0 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] conn(0x7f76740ad4b0 msgr2=0x7f76740af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] conn(0x7f76740ad4b0 0x7f76740af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f76740a3cd0 msgr2=0x7f76740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f76740a3cd0 0x7f76740a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7688078f40 tx=0x7f767800b3c0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7688101940 msgr2=0x7f7688199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 --2- 172.21.15.72:0/1453055978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7688101940 0x7f7688199180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7684000c00 tx=0x7f76840047b0 comp rx=0 tx=0).stop 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768dd75700 1 -- 172.21.15.72:0/1453055978 reap_dead start 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 shutdown_connections 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 >> 172.21.15.72:0/1453055978 conn(0x7f76880788f0 msgr2=0x7f768806a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:44.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 shutdown_connections 2024-04-25T13:11:44.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:44.655+0000 7f768f7d8700 1 -- 172.21.15.72:0/1453055978 wait complete. 2024-04-25T13:11:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:44 smithi193 ceph-mon[32743]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:11:44.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:44 smithi164 ceph-mon[32418]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:11:45.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.032+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/2636346798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3e8108ad0 msgr2=0x7fd3e8108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.032+0000 7fd3ef6d5700 1 --2- 172.21.15.72:0/2636346798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3e8108ad0 0x7fd3e8108eb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0009990 tx=0x7fd3e0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.032+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/2636346798 shutdown_connections 2024-04-25T13:11:45.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.032+0000 7fd3ef6d5700 1 --2- 172.21.15.72:0/2636346798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e81019d0 0x7fd3e810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.032+0000 7fd3ef6d5700 1 --2- 172.21.15.72:0/2636346798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3e8108ad0 0x7fd3e8108eb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.032+0000 7fd3ef6d5700 1 --2- 172.21.15.72:0/2636346798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e8101030 0x7fd3e8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.032+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/2636346798 >> 172.21.15.72:0/2636346798 conn(0x7fd3e8077fb0 msgr2=0x7fd3e80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.032+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/2636346798 shutdown_connections 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.032+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/2636346798 wait complete. 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ef6d5700 1 Processor -- start 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ef6d5700 1 -- start start 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ef6d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e8101030 0x7fd3e8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ef6d5700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e81019d0 0x7fd3e8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ef6d5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3e8108ad0 0x7fd3e819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ef6d5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8112bc0 con 0x7fd3e8108ad0 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ef6d5700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8112a40 con 0x7fd3e81019d0 2024-04-25T13:11:45.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ef6d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8112d40 con 0x7fd3e8101030 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3edc72700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3e8108ad0 0x7fd3e819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3edc72700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3e8108ad0 0x7fd3e819d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34308/0 (socket says 172.21.15.72:34308) 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3edc72700 1 -- 172.21.15.72:0/3025752139 learned_addr learned my addr 172.21.15.72:0/3025752139 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ed471700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e8101030 0x7fd3e8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3edc72700 1 -- 172.21.15.72:0/3025752139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e8101030 msgr2=0x7fd3e8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ecc70700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e81019d0 0x7fd3e8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3edc72700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e8101030 0x7fd3e8199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3edc72700 1 -- 172.21.15.72:0/3025752139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e81019d0 msgr2=0x7fd3e8199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3edc72700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e81019d0 0x7fd3e8199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3edc72700 1 -- 172.21.15.72:0/3025752139 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e819e120 con 0x7fd3e8108ad0 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.033+0000 7fd3ecc70700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e81019d0 0x7fd3e8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:45.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.034+0000 7fd3edc72700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3e8108ad0 0x7fd3e819d9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4007ee0 tx=0x7fd3e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:45.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.034+0000 7fd3de7fc700 1 -- 172.21.15.72:0/3025752139 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400dbd0 con 0x7fd3e8108ad0 2024-04-25T13:11:45.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.034+0000 7fd3de7fc700 1 -- 172.21.15.72:0/3025752139 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e400dd70 con 0x7fd3e8108ad0 2024-04-25T13:11:45.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.034+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e819e410 con 0x7fd3e8108ad0 2024-04-25T13:11:45.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.034+0000 7fd3de7fc700 1 -- 172.21.15.72:0/3025752139 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4005990 con 0x7fd3e8108ad0 2024-04-25T13:11:45.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.034+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e81a5c90 con 0x7fd3e8108ad0 2024-04-25T13:11:45.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.034+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3e8063150 con 0x7fd3e8108ad0 2024-04-25T13:11:45.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.036+0000 7fd3de7fc700 1 -- 172.21.15.72:0/3025752139 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd3e401a020 con 0x7fd3e8108ad0 2024-04-25T13:11:45.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.036+0000 7fd3de7fc700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3d40a3cd0 0x7fd3d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.037+0000 7fd3ed471700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3d40a3cd0 0x7fd3d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.037+0000 7fd3de7fc700 1 -- 172.21.15.72:0/3025752139 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd3e40d02c0 con 0x7fd3e8108ad0 2024-04-25T13:11:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.037+0000 7fd3de7fc700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] conn(0x7fd3d40ad4b0 0x7fd3d40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.037+0000 7fd3de7fc700 1 -- 172.21.15.72:0/3025752139 --> [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3d40aff80 con 0x7fd3d40ad4b0 2024-04-25T13:11:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.037+0000 7fd3de7fc700 1 -- 172.21.15.72:0/3025752139 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd3e40d07a0 con 0x7fd3e8108ad0 2024-04-25T13:11:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.037+0000 7fd3ed471700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3d40a3cd0 0x7fd3d40a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000b5c0 tx=0x7fd3e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.037+0000 7fd3ecc70700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] conn(0x7fd3d40ad4b0 0x7fd3d40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.038+0000 7fd3ecc70700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] conn(0x7fd3d40ad4b0 0x7fd3d40af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.038+0000 7fd3de7fc700 1 -- 172.21.15.72:0/3025752139 <== osd.4 v2:172.21.15.164:6808/1819655779 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3d40aff80 con 0x7fd3d40ad4b0 2024-04-25T13:11:45.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.055+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 --> [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd3e80635d0 con 0x7fd3d40ad4b0 2024-04-25T13:11:45.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.055+0000 7fd3de7fc700 1 -- 172.21.15.72:0/3025752139 <== osd.4 v2:172.21.15.164:6808/1819655779 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd3e80635d0 con 0x7fd3d40ad4b0 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 >> [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] conn(0x7fd3d40ad4b0 msgr2=0x7fd3d40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] conn(0x7fd3d40ad4b0 0x7fd3d40af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3d40a3cd0 msgr2=0x7fd3d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd3d40a3cd0 0x7fd3d40a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000b5c0 tx=0x7fd3e0005fb0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3e8108ad0 msgr2=0x7fd3e819d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 --2- 172.21.15.72:0/3025752139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3e8108ad0 0x7fd3e819d9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4007ee0 tx=0x7fd3e400d3b0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3edc72700 1 -- 172.21.15.72:0/3025752139 reap_dead start 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 shutdown_connections 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 >> 172.21.15.72:0/3025752139 conn(0x7fd3e8077fb0 msgr2=0x7fd3e80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:45.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 shutdown_connections 2024-04-25T13:11:45.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.056+0000 7fd3ef6d5700 1 -- 172.21.15.72:0/3025752139 wait complete. 2024-04-25T13:11:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 -- 172.21.15.72:0/2138179553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0f406bb30 msgr2=0x7fb0f410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 --2- 172.21.15.72:0/2138179553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0f406bb30 0x7fb0f410f900 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400b0a0 tx=0x7fb0e400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 -- 172.21.15.72:0/2138179553 shutdown_connections 2024-04-25T13:11:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 --2- 172.21.15.72:0/2138179553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0f406bb30 0x7fb0f410f900 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 --2- 172.21.15.72:0/2138179553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f4108a30 0x7fb0f4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 --2- 172.21.15.72:0/2138179553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f406b190 0x7fb0f406b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 -- 172.21.15.72:0/2138179553 >> 172.21.15.72:0/2138179553 conn(0x7fb0f4078910 msgr2=0x7fb0f4078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 -- 172.21.15.72:0/2138179553 shutdown_connections 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 -- 172.21.15.72:0/2138179553 wait complete. 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 Processor -- start 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.261+0000 7fb0fb51d700 1 -- start start 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0fb51d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f406b190 0x7fb0f4199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0fb51d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f406bb30 0x7fb0f4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0fb51d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0f4108a30 0x7fb0f419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0fb51d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4112c70 con 0x7fb0f4108a30 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0fb51d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4112af0 con 0x7fb0f406b190 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0fb51d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4112df0 con 0x7fb0f406bb30 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f9aba700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0f4108a30 0x7fb0f419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f9aba700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0f4108a30 0x7fb0f419d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34336/0 (socket says 172.21.15.72:34336) 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f9aba700 1 -- 172.21.15.72:0/3171667176 learned_addr learned my addr 172.21.15.72:0/3171667176 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f92b9700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f406b190 0x7fb0f4199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f8ab8700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f406bb30 0x7fb0f4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f9aba700 1 -- 172.21.15.72:0/3171667176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f406bb30 msgr2=0x7fb0f4199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f9aba700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f406bb30 0x7fb0f4199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f9aba700 1 -- 172.21.15.72:0/3171667176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f406b190 msgr2=0x7fb0f4199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f9aba700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f406b190 0x7fb0f4199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f9aba700 1 -- 172.21.15.72:0/3171667176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f419e130 con 0x7fb0f4108a30 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f92b9700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f406b190 0x7fb0f4199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f8ab8700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f406bb30 0x7fb0f4199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f9aba700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0f4108a30 0x7fb0f419d9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400b830 tx=0x7fb0e4004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f27fc700 1 -- 172.21.15.72:0/3171667176 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4021070 con 0x7fb0f4108a30 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f27fc700 1 -- 172.21.15.72:0/3171667176 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e4003c00 con 0x7fb0f4108a30 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f419e3c0 con 0x7fb0f4108a30 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0f27fc700 1 -- 172.21.15.72:0/3171667176 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4004730 con 0x7fb0f4108a30 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.262+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f41a5ca0 con 0x7fb0f4108a30 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.263+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0f40631d0 con 0x7fb0f4108a30 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.264+0000 7fb0f27fc700 1 -- 172.21.15.72:0/3171667176 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb0e4003db0 con 0x7fb0f4108a30 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.265+0000 7fb0f27fc700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0dc0a3d50 0x7fb0dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.265+0000 7fb0f92b9700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0dc0a3d50 0x7fb0dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.265+0000 7fb0f27fc700 1 -- 172.21.15.72:0/3171667176 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb0e40d1950 con 0x7fb0f4108a30 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.265+0000 7fb0f27fc700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] conn(0x7fb0dc0ad530 0x7fb0dc0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.265+0000 7fb0f27fc700 1 -- 172.21.15.72:0/3171667176 --> [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0e401ce50 con 0x7fb0dc0ad530 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.265+0000 7fb0f27fc700 1 -- 172.21.15.72:0/3171667176 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fb0e40d1e30 con 0x7fb0f4108a30 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.265+0000 7fb0f92b9700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0dc0a3d50 0x7fb0dc0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec005fd0 tx=0x7fb0ec005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:45.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.266+0000 7fb0f8ab8700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] conn(0x7fb0dc0ad530 0x7fb0dc0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.266+0000 7fb0f8ab8700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] conn(0x7fb0dc0ad530 0x7fb0dc0af950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.267+0000 7fb0f27fc700 1 -- 172.21.15.72:0/3171667176 <== osd.10 v2:172.21.15.164:6824/1853012500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0e401ce50 con 0x7fb0dc0ad530 2024-04-25T13:11:45.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.288+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 --> [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb0f4063650 con 0x7fb0dc0ad530 2024-04-25T13:11:45.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.288+0000 7fb0f27fc700 1 -- 172.21.15.72:0/3171667176 <== osd.10 v2:172.21.15.164:6824/1853012500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb0f4063650 con 0x7fb0dc0ad530 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.288+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 >> [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] conn(0x7fb0dc0ad530 msgr2=0x7fb0dc0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.288+0000 7fb0fb51d700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] conn(0x7fb0dc0ad530 0x7fb0dc0af950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.289+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0dc0a3d50 msgr2=0x7fb0dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.289+0000 7fb0fb51d700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0dc0a3d50 0x7fb0dc0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec005fd0 tx=0x7fb0ec005ee0 comp rx=0 tx=0).stop 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.289+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0f4108a30 msgr2=0x7fb0f419d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.289+0000 7fb0fb51d700 1 --2- 172.21.15.72:0/3171667176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0f4108a30 0x7fb0f419d9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400b830 tx=0x7fb0e4004d90 comp rx=0 tx=0).stop 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.289+0000 7fb0f9aba700 1 -- 172.21.15.72:0/3171667176 reap_dead start 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.289+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 shutdown_connections 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.289+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 >> 172.21.15.72:0/3171667176 conn(0x7fb0f4078910 msgr2=0x7fb0f40ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.289+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 shutdown_connections 2024-04-25T13:11:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:45.289+0000 7fb0fb51d700 1 -- 172.21.15.72:0/3171667176 wait complete. 2024-04-25T13:11:46.034 INFO:teuthology.orchestra.run.smithi072.stdout:77309411393 2024-04-25T13:11:46.034 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T13:11:46.729 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:46 smithi072 ceph-mon[29199]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:11:46.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:46 smithi193 ceph-mon[32743]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:11:46.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:46 smithi164 ceph-mon[32418]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:11:47.007 INFO:teuthology.orchestra.run.smithi072.stdout:34359738444 2024-04-25T13:11:47.007 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T13:11:47.640 INFO:teuthology.orchestra.run.smithi072.stdout:167503724596 2024-04-25T13:11:47.641 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.197+0000 7f86f85aa700 1 -- 172.21.15.72:0/2454635985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86f0108a80 msgr2=0x7f86f0108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.197+0000 7f86f85aa700 1 --2- 172.21.15.72:0/2454635985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86f0108a80 0x7f86f0108e60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009990 tx=0x7f86ec009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.197+0000 7f86f85aa700 1 -- 172.21.15.72:0/2454635985 shutdown_connections 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.197+0000 7f86f85aa700 1 --2- 172.21.15.72:0/2454635985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f01034b0 0x7f86f010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.197+0000 7f86f85aa700 1 --2- 172.21.15.72:0/2454635985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86f0108a80 0x7f86f0108e60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.197+0000 7f86f85aa700 1 --2- 172.21.15.72:0/2454635985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f0102af0 0x7f86f0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.197+0000 7f86f85aa700 1 -- 172.21.15.72:0/2454635985 >> 172.21.15.72:0/2454635985 conn(0x7f86f0079ad0 msgr2=0x7f86f0079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.197+0000 7f86f85aa700 1 -- 172.21.15.72:0/2454635985 shutdown_connections 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.197+0000 7f86f85aa700 1 -- 172.21.15.72:0/2454635985 wait complete. 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f85aa700 1 Processor -- start 2024-04-25T13:11:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f85aa700 1 -- start start 2024-04-25T13:11:47.724 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f85aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f0102af0 0x7f86f0199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:47.724 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f85aa700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86f01034b0 0x7f86f0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:47.724 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f85aa700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f0108a80 0x7f86f019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:47.724 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f85aa700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112c30 con 0x7f86f01034b0 2024-04-25T13:11:47.724 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f85aa700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112ab0 con 0x7f86f0108a80 2024-04-25T13:11:47.724 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f85aa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112db0 con 0x7f86f0102af0 2024-04-25T13:11:47.724 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f5b45700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86f01034b0 0x7f86f0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:47.724 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f5b45700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86f01034b0 0x7f86f0199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56928/0 (socket says 172.21.15.72:56928) 2024-04-25T13:11:47.724 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f5b45700 1 -- 172.21.15.72:0/2208065228 learned_addr learned my addr 172.21.15.72:0/2208065228 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.198+0000 7f86f6346700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f0102af0 0x7f86f0199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86f6b47700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f0108a80 0x7f86f019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86f5b45700 1 -- 172.21.15.72:0/2208065228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f0102af0 msgr2=0x7f86f0199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86f5b45700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f0102af0 0x7f86f0199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86f5b45700 1 -- 172.21.15.72:0/2208065228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f0108a80 msgr2=0x7f86f019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86f5b45700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f0108a80 0x7f86f019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86f5b45700 1 -- 172.21.15.72:0/2208065228 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f019e110 con 0x7f86f01034b0 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86f5b45700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86f01034b0 0x7f86f0199640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f86e000c970 tx=0x7f86e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86e77fe700 1 -- 172.21.15.72:0/2208065228 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e00078d0 con 0x7f86f01034b0 2024-04-25T13:11:47.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f019e400 con 0x7f86f01034b0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86e77fe700 1 -- 172.21.15.72:0/2208065228 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e0018480 con 0x7f86f01034b0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86e77fe700 1 -- 172.21.15.72:0/2208065228 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e000f6b0 con 0x7f86f01034b0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.199+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a5c80 con 0x7f86f01034b0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.200+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86f0063150 con 0x7f86f01034b0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.201+0000 7f86e77fe700 1 -- 172.21.15.72:0/2208065228 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86e0018930 con 0x7f86f01034b0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.202+0000 7f86e77fe700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f86dc0a3cd0 0x7f86dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.202+0000 7f86f6346700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f86dc0a3cd0 0x7f86dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.202+0000 7f86e77fe700 1 -- 172.21.15.72:0/2208065228 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f86e00d0910 con 0x7f86f01034b0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.202+0000 7f86e77fe700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] conn(0x7f86dc0ad4b0 0x7f86dc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.202+0000 7f86e77fe700 1 -- 172.21.15.72:0/2208065228 --> [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86dc0aff80 con 0x7f86dc0ad4b0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.202+0000 7f86e77fe700 1 -- 172.21.15.72:0/2208065228 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f86e00d0df0 con 0x7f86f01034b0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.202+0000 7f86f6346700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f86dc0a3cd0 0x7f86dc0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00b5c0 tx=0x7f86ec005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.202+0000 7f86f6b47700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] conn(0x7f86dc0ad4b0 0x7f86dc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.203+0000 7f86f6b47700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] conn(0x7f86dc0ad4b0 0x7f86dc0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.204+0000 7f86e77fe700 1 -- 172.21.15.72:0/2208065228 <== osd.1 v2:172.21.15.164:6800/4084359931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86dc0aff80 con 0x7f86dc0ad4b0 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.227+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 --> [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86f00635d0 con 0x7f86dc0ad4b0 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86e77fe700 1 -- 172.21.15.72:0/2208065228 <== osd.1 v2:172.21.15.164:6800/4084359931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f86f00635d0 con 0x7f86dc0ad4b0 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 >> [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] conn(0x7f86dc0ad4b0 msgr2=0x7f86dc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] conn(0x7f86dc0ad4b0 0x7f86dc0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f86dc0a3cd0 msgr2=0x7f86dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f86dc0a3cd0 0x7f86dc0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00b5c0 tx=0x7f86ec005960 comp rx=0 tx=0).stop 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86f01034b0 msgr2=0x7f86f0199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 --2- 172.21.15.72:0/2208065228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86f01034b0 0x7f86f0199640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f86e000c970 tx=0x7f86e000cc80 comp rx=0 tx=0).stop 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f6b47700 1 -- 172.21.15.72:0/2208065228 reap_dead start 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 shutdown_connections 2024-04-25T13:11:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 >> 172.21.15.72:0/2208065228 conn(0x7f86f0079ad0 msgr2=0x7f86f0101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:47.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 shutdown_connections 2024-04-25T13:11:47.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:46.228+0000 7f86f85aa700 1 -- 172.21.15.72:0/2208065228 wait complete. 2024-04-25T13:11:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:55834574920 2024-04-25T13:11:48.496 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T13:11:48.516 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:48 smithi072 ceph-mon[29199]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:11:48.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:48 smithi193 ceph-mon[32743]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:11:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:48 smithi164 ceph-mon[32418]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:11:49.083 INFO:teuthology.orchestra.run.smithi072.stdout:141733920823 2024-04-25T13:11:49.084 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 -- 172.21.15.72:0/1182142165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb41017d0 msgr2=0x7fefb4108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 --2- 172.21.15.72:0/1182142165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb41017d0 0x7fefb4108820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fefa80099c0 tx=0x7fefa8009cd0 comp rx=0 tx=0).stop 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 -- 172.21.15.72:0/1182142165 shutdown_connections 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 --2- 172.21.15.72:0/1182142165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb4108f00 0x7fefb410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 --2- 172.21.15.72:0/1182142165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb4100e20 0x7fefb4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 --2- 172.21.15.72:0/1182142165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb41017d0 0x7fefb4108820 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 -- 172.21.15.72:0/1182142165 >> 172.21.15.72:0/1182142165 conn(0x7fefb4077b40 msgr2=0x7fefb40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 -- 172.21.15.72:0/1182142165 shutdown_connections 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 -- 172.21.15.72:0/1182142165 wait complete. 2024-04-25T13:11:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.379+0000 7fefba058700 1 Processor -- start 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefba058700 1 -- start start 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefba058700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb4100e20 0x7fefb4102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefba058700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb41017d0 0x7fefb4102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefba058700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb4108f00 0x7fefb41030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefba058700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4112bf0 con 0x7fefb4108f00 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefba058700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4112a70 con 0x7fefb4100e20 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefba058700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4112d70 con 0x7fefb41017d0 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb3fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb4108f00 0x7fefb41030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb3fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb4108f00 0x7fefb41030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56960/0 (socket says 172.21.15.72:56960) 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb3fff700 1 -- 172.21.15.72:0/4099773737 learned_addr learned my addr 172.21.15.72:0/4099773737 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb2ffd700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb41017d0 0x7fefb4102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb37fe700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb4100e20 0x7fefb4102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb3fff700 1 -- 172.21.15.72:0/4099773737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb41017d0 msgr2=0x7fefb4102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:49.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb3fff700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb41017d0 0x7fefb4102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb3fff700 1 -- 172.21.15.72:0/4099773737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb4100e20 msgr2=0x7fefb4102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb3fff700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb4100e20 0x7fefb4102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb3fff700 1 -- 172.21.15.72:0/4099773737 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb41a7670 con 0x7fefb4108f00 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.380+0000 7fefb3fff700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb4108f00 0x7fefb41030b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fefac007ee0 tx=0x7fefac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.381+0000 7fefb0ff9700 1 -- 172.21.15.72:0/4099773737 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac00dbd0 con 0x7fefb4108f00 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.381+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb41a7960 con 0x7fefb4108f00 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.381+0000 7fefb0ff9700 1 -- 172.21.15.72:0/4099773737 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefac00dd70 con 0x7fefb4108f00 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.381+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41a7e70 con 0x7fefb4108f00 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.381+0000 7fefb0ff9700 1 -- 172.21.15.72:0/4099773737 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac005990 con 0x7fefb4108f00 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.381+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fefb40630d0 con 0x7fefb4108f00 2024-04-25T13:11:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.383+0000 7fefb0ff9700 1 -- 172.21.15.72:0/4099773737 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fefac01a020 con 0x7fefb4108f00 2024-04-25T13:11:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.383+0000 7fefb0ff9700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fef9c0a3cd0 0x7fef9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.384+0000 7fefb37fe700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fef9c0a3cd0 0x7fef9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.384+0000 7fefb0ff9700 1 -- 172.21.15.72:0/4099773737 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fefac0d0540 con 0x7fefb4108f00 2024-04-25T13:11:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.384+0000 7fefb0ff9700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] conn(0x7fef9c0ad4b0 0x7fef9c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.384+0000 7fefb0ff9700 1 -- 172.21.15.72:0/4099773737 --> [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefac01de50 con 0x7fef9c0ad4b0 2024-04-25T13:11:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.384+0000 7fefb0ff9700 1 -- 172.21.15.72:0/4099773737 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fefac0d0a20 con 0x7fefb4108f00 2024-04-25T13:11:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.384+0000 7fefb37fe700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fef9c0a3cd0 0x7fef9c0a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005950 tx=0x7fefa40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.384+0000 7fefb2ffd700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] conn(0x7fef9c0ad4b0 0x7fef9c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.385+0000 7fefb2ffd700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] conn(0x7fef9c0ad4b0 0x7fef9c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.386+0000 7fefb0ff9700 1 -- 172.21.15.72:0/4099773737 <== osd.7 v2:172.21.15.164:6816/3610097866 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefac01de50 con 0x7fef9c0ad4b0 2024-04-25T13:11:49.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.402+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 --> [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fefb40634d0 con 0x7fef9c0ad4b0 2024-04-25T13:11:49.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.402+0000 7fefb0ff9700 1 -- 172.21.15.72:0/4099773737 <== osd.7 v2:172.21.15.164:6816/3610097866 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fefb40634d0 con 0x7fef9c0ad4b0 2024-04-25T13:11:49.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 >> [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] conn(0x7fef9c0ad4b0 msgr2=0x7fef9c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:49.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] conn(0x7fef9c0ad4b0 0x7fef9c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:49.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fef9c0a3cd0 msgr2=0x7fef9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:49.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fef9c0a3cd0 0x7fef9c0a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005950 tx=0x7fefa40058e0 comp rx=0 tx=0).stop 2024-04-25T13:11:49.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb4108f00 msgr2=0x7fefb41030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:49.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 --2- 172.21.15.72:0/4099773737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb4108f00 0x7fefb41030b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fefac007ee0 tx=0x7fefac00d3b0 comp rx=0 tx=0).stop 2024-04-25T13:11:49.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefb3fff700 1 -- 172.21.15.72:0/4099773737 reap_dead start 2024-04-25T13:11:49.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 shutdown_connections 2024-04-25T13:11:49.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 >> 172.21.15.72:0/4099773737 conn(0x7fefb4077b40 msgr2=0x7fefb40ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:49.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 shutdown_connections 2024-04-25T13:11:49.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:49.403+0000 7fefba058700 1 -- 172.21.15.72:0/4099773737 wait complete. 2024-04-25T13:11:50.036 INFO:teuthology.orchestra.run.smithi072.stdout:223338299433 2024-04-25T13:11:50.036 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T13:11:50.515 INFO:teuthology.orchestra.run.smithi072.stdout:201863462959 2024-04-25T13:11:50.515 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T13:11:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:50 smithi193 ceph-mon[32743]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:11:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:50 smithi072 ceph-mon[29199]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:11:50.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:50 smithi164 ceph-mon[32418]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:11:51.178 INFO:teuthology.orchestra.run.smithi072.stdout:322122547227 2024-04-25T13:11:51.179 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T13:11:51.186 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:51.322 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:51.322 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:52.356 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:52.357 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:52.358 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:52.584 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:52 smithi072 ceph-mon[29199]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:11:52.593 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:52.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:52 smithi193 ceph-mon[32743]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:11:52.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:52 smithi164 ceph-mon[32418]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:11:54.673 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:54 smithi072 ceph-mon[29199]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:11:54.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:54 smithi193 ceph-mon[32743]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:11:54.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:54 smithi164 ceph-mon[32418]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:11:56.657 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:56 smithi072 ceph-mon[29199]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:56.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:56 smithi193 ceph-mon[32743]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:56.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:56 smithi164 ceph-mon[32418]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:57.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.081+0000 7f14b8473700 1 -- 172.21.15.72:0/2663417412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b0101da0 msgr2=0x7f14b010f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.081+0000 7f14b8473700 1 --2- 172.21.15.72:0/2663417412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b0101da0 0x7f14b010f810 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f14a800b0a0 tx=0x7f14a800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.081+0000 7f14b8473700 1 -- 172.21.15.72:0/2663417412 shutdown_connections 2024-04-25T13:11:57.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.081+0000 7f14b8473700 1 --2- 172.21.15.72:0/2663417412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b0101da0 0x7f14b010f810 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.081+0000 7f14b8473700 1 --2- 172.21.15.72:0/2663417412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b0107b00 0x7f14b0107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.081+0000 7f14b8473700 1 --2- 172.21.15.72:0/2663417412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14b01013e0 0x7f14b0101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.081+0000 7f14b8473700 1 -- 172.21.15.72:0/2663417412 >> 172.21.15.72:0/2663417412 conn(0x7f14b0076c60 msgr2=0x7f14b0077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:57.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.082+0000 7f14b8473700 1 -- 172.21.15.72:0/2663417412 shutdown_connections 2024-04-25T13:11:57.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.082+0000 7f14b8473700 1 -- 172.21.15.72:0/2663417412 wait complete. 2024-04-25T13:11:57.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.082+0000 7f14b8473700 1 Processor -- start 2024-04-25T13:11:57.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.082+0000 7f14b8473700 1 -- start start 2024-04-25T13:11:57.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.082+0000 7f14b8473700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b01013e0 0x7f14b0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:57.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.082+0000 7f14b8473700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14b0101da0 0x7f14b0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:57.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b8473700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b0107b00 0x7f14b019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:57.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b8473700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0112d10 con 0x7f14b01013e0 2024-04-25T13:11:57.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b8473700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0112b90 con 0x7f14b0107b00 2024-04-25T13:11:57.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b8473700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0112e90 con 0x7f14b0101da0 2024-04-25T13:11:57.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b620f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b01013e0 0x7f14b0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:57.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b620f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b01013e0 0x7f14b0199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37032/0 (socket says 172.21.15.72:37032) 2024-04-25T13:11:57.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b620f700 1 -- 172.21.15.72:0/3531400289 learned_addr learned my addr 172.21.15.72:0/3531400289 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:57.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b5a0e700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14b0101da0 0x7f14b0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:57.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b620f700 1 -- 172.21.15.72:0/3531400289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14b0101da0 msgr2=0x7f14b0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b620f700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14b0101da0 0x7f14b0199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b620f700 1 -- 172.21.15.72:0/3531400289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b0107b00 msgr2=0x7f14b019dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b620f700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b0107b00 0x7f14b019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b620f700 1 -- 172.21.15.72:0/3531400289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b019e210 con 0x7f14b01013e0 2024-04-25T13:11:57.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b620f700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b01013e0 0x7f14b0199200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00b500 tx=0x7f14ac00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:57.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14a77fe700 1 -- 172.21.15.72:0/3531400289 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac015020 con 0x7f14b01013e0 2024-04-25T13:11:57.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14a77fe700 1 -- 172.21.15.72:0/3531400289 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ac011ce0 con 0x7f14b01013e0 2024-04-25T13:11:57.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b019e500 con 0x7f14b01013e0 2024-04-25T13:11:57.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14a77fe700 1 -- 172.21.15.72:0/3531400289 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac010d60 con 0x7f14b01013e0 2024-04-25T13:11:57.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.083+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b01a5e00 con 0x7f14b01013e0 2024-04-25T13:11:57.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.084+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b0069730 con 0x7f14b01013e0 2024-04-25T13:11:57.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.085+0000 7f14a77fe700 1 -- 172.21.15.72:0/3531400289 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f14ac018020 con 0x7f14b01013e0 2024-04-25T13:11:57.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.086+0000 7f14a77fe700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f149c0a3d50 0x7f149c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:57.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.086+0000 7f14b5a0e700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f149c0a3d50 0x7f149c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:57.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.086+0000 7f14a77fe700 1 -- 172.21.15.72:0/3531400289 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f14ac0d00e0 con 0x7f14b01013e0 2024-04-25T13:11:57.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.086+0000 7f14b5a0e700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f149c0a3d50 0x7f149c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006fd0 tx=0x7f14a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:57.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.090+0000 7f14a77fe700 1 -- 172.21.15.72:0/3531400289 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14ac01f020 con 0x7f14b01013e0 2024-04-25T13:11:57.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.296+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f14b002df80 con 0x7f14b01013e0 2024-04-25T13:11:57.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.296+0000 7f14a77fe700 1 -- 172.21.15.72:0/3531400289 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14ac09cb20 con 0x7f14b01013e0 2024-04-25T13:11:57.296 INFO:teuthology.orchestra.run.smithi072.stdout:77309411395 2024-04-25T13:11:57.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f149c0a3d50 msgr2=0x7f149c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f149c0a3d50 0x7f149c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006fd0 tx=0x7f14a0008040 comp rx=0 tx=0).stop 2024-04-25T13:11:57.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b01013e0 msgr2=0x7f14b0199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b01013e0 0x7f14b0199200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00b500 tx=0x7f14ac00b8c0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 shutdown_connections 2024-04-25T13:11:57.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b0107b00 0x7f14b019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f149c0a3d50 0x7f149c0a6210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b01013e0 0x7f14b0199200 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 --2- 172.21.15.72:0/3531400289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14b0101da0 0x7f14b0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 >> 172.21.15.72:0/3531400289 conn(0x7f14b0076c60 msgr2=0x7f14b0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:57.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 shutdown_connections 2024-04-25T13:11:57.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.299+0000 7f14b8473700 1 -- 172.21.15.72:0/3531400289 wait complete. 2024-04-25T13:11:57.589 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:57 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3531400289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:11:57.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.690+0000 7f79850d0700 1 -- 172.21.15.72:0/2583817013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7980075590 msgr2=0x7f798010ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.690+0000 7f79850d0700 1 --2- 172.21.15.72:0/2583817013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7980075590 0x7f798010ef70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7974009990 tx=0x7f7974009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.690+0000 7f79850d0700 1 -- 172.21.15.72:0/2583817013 shutdown_connections 2024-04-25T13:11:57.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.690+0000 7f79850d0700 1 --2- 172.21.15.72:0/2583817013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f798010f4b0 0x7f79801118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.690+0000 7f79850d0700 1 --2- 172.21.15.72:0/2583817013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980074c70 0x7f7980075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.690+0000 7f79850d0700 1 --2- 172.21.15.72:0/2583817013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7980075590 0x7f798010ef70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.690+0000 7f79850d0700 1 -- 172.21.15.72:0/2583817013 >> 172.21.15.72:0/2583817013 conn(0x7f7980079c00 msgr2=0x7f798007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:57.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.690+0000 7f79850d0700 1 -- 172.21.15.72:0/2583817013 shutdown_connections 2024-04-25T13:11:57.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.690+0000 7f79850d0700 1 -- 172.21.15.72:0/2583817013 wait complete. 2024-04-25T13:11:57.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.691+0000 7f79850d0700 1 Processor -- start 2024-04-25T13:11:57.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.691+0000 7f79850d0700 1 -- start start 2024-04-25T13:11:57.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.691+0000 7f79850d0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980074c70 0x7f798019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:57.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.691+0000 7f79850d0700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7980075590 0x7f798019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:57.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.691+0000 7f79850d0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798010f4b0 0x7f798019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:57.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.691+0000 7f79850d0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7980114e40 con 0x7f798010f4b0 2024-04-25T13:11:57.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.691+0000 7f79850d0700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7980114cc0 con 0x7f7980075590 2024-04-25T13:11:57.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.691+0000 7f79850d0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7980114fc0 con 0x7f7980074c70 2024-04-25T13:11:57.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797f59e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798010f4b0 0x7f798019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:57.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797f59e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798010f4b0 0x7f798019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37060/0 (socket says 172.21.15.72:37060) 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797e59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7980075590 0x7f798019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797f59e700 1 -- 172.21.15.72:0/2717001182 learned_addr learned my addr 172.21.15.72:0/2717001182 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797ed9d700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980074c70 0x7f798019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797f59e700 1 -- 172.21.15.72:0/2717001182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980074c70 msgr2=0x7f798019b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797f59e700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980074c70 0x7f798019b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797f59e700 1 -- 172.21.15.72:0/2717001182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7980075590 msgr2=0x7f798019b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797f59e700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7980075590 0x7f798019b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797f59e700 1 -- 172.21.15.72:0/2717001182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79801a04a0 con 0x7f798010f4b0 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797e59c700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7980075590 0x7f798019b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f797f59e700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798010f4b0 0x7f798019fd60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7978007ee0 tx=0x7f797800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:57.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f7967fff700 1 -- 172.21.15.72:0/2717001182 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978004bb0 con 0x7f798010f4b0 2024-04-25T13:11:57.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79801a0790 con 0x7f798010f4b0 2024-04-25T13:11:57.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f7967fff700 1 -- 172.21.15.72:0/2717001182 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7978004d50 con 0x7f798010f4b0 2024-04-25T13:11:57.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f7967fff700 1 -- 172.21.15.72:0/2717001182 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978005920 con 0x7f798010f4b0 2024-04-25T13:11:57.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.692+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79801a8010 con 0x7f798010f4b0 2024-04-25T13:11:57.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.693+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79800762c0 con 0x7f798010f4b0 2024-04-25T13:11:57.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.695+0000 7f7967fff700 1 -- 172.21.15.72:0/2717001182 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f797801a020 con 0x7f798010f4b0 2024-04-25T13:11:57.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.696+0000 7f7967fff700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f79680a3d50 0x7f79680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:57.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.696+0000 7f797ed9d700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f79680a3d50 0x7f79680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:57.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.696+0000 7f7967fff700 1 -- 172.21.15.72:0/2717001182 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f79780d00c0 con 0x7f798010f4b0 2024-04-25T13:11:57.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.696+0000 7f797ed9d700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f79680a3d50 0x7f79680a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7970009a10 tx=0x7f7970009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:57.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.699+0000 7f7967fff700 1 -- 172.21.15.72:0/2717001182 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7978010070 con 0x7f798010f4b0 2024-04-25T13:11:57.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:57 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3531400289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:11:57.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:57 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3531400289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:11:57.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.906+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f798002df80 con 0x7f798010f4b0 2024-04-25T13:11:57.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.907+0000 7f7967fff700 1 -- 172.21.15.72:0/2717001182 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f797809caf0 con 0x7f798010f4b0 2024-04-25T13:11:57.907 INFO:teuthology.orchestra.run.smithi072.stdout:55834574922 2024-04-25T13:11:57.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f79680a3d50 msgr2=0x7f79680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f79680a3d50 0x7f79680a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7970009a10 tx=0x7f7970009190 comp rx=0 tx=0).stop 2024-04-25T13:11:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798010f4b0 msgr2=0x7f798019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798010f4b0 0x7f798019fd60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7978007ee0 tx=0x7f797800d3b0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 shutdown_connections 2024-04-25T13:11:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f79680a3d50 0x7f79680a6210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798010f4b0 0x7f798019fd60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980074c70 0x7f798019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 --2- 172.21.15.72:0/2717001182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7980075590 0x7f798019b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 >> 172.21.15.72:0/2717001182 conn(0x7f7980079c00 msgr2=0x7f7980101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 shutdown_connections 2024-04-25T13:11:57.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:57.909+0000 7f79850d0700 1 -- 172.21.15.72:0/2717001182 wait complete. 2024-04-25T13:11:58.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:58 smithi072 ceph-mon[29199]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:58.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:58 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2717001182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:11:58.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:58 smithi193 ceph-mon[32743]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:58.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:58 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2717001182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:11:58.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:58 smithi164 ceph-mon[32418]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:58.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:58 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2717001182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:11:59.149 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411395 for osd.2 2024-04-25T13:11:59.149 DEBUG:teuthology.parallel:result is None 2024-04-25T13:11:59.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:11:59 smithi193 ceph-mon[32743]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:59.739 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:11:59 smithi072 ceph-mon[29199]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:59.755 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:59.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.760+0000 7f047a817700 1 -- 172.21.15.72:0/2011199200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04741034b0 msgr2=0x7f047410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:59.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.760+0000 7f047a817700 1 --2- 172.21.15.72:0/2011199200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04741034b0 0x7f047410f8e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f046c00b0a0 tx=0x7f046c00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.760+0000 7f047a817700 1 -- 172.21.15.72:0/2011199200 shutdown_connections 2024-04-25T13:11:59.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.760+0000 7f047a817700 1 --2- 172.21.15.72:0/2011199200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04741034b0 0x7f047410f8e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.760+0000 7f047a817700 1 --2- 172.21.15.72:0/2011199200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0474108a80 0x7f0474108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.760+0000 7f047a817700 1 --2- 172.21.15.72:0/2011199200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0474102af0 0x7f0474102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.760+0000 7f047a817700 1 -- 172.21.15.72:0/2011199200 >> 172.21.15.72:0/2011199200 conn(0x7f0474079ad0 msgr2=0x7f0474079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:59.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.760+0000 7f047a817700 1 -- 172.21.15.72:0/2011199200 shutdown_connections 2024-04-25T13:11:59.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.760+0000 7f047a817700 1 -- 172.21.15.72:0/2011199200 wait complete. 2024-04-25T13:11:59.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.761+0000 7f047a817700 1 Processor -- start 2024-04-25T13:11:59.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.761+0000 7f047a817700 1 -- start start 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.761+0000 7f047a817700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0474102af0 0x7f0474074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.761+0000 7f047a817700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04741034b0 0x7f0474075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.761+0000 7f047a817700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0474108a80 0x7f04740757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.761+0000 7f047a817700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0474112c10 con 0x7f0474108a80 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.761+0000 7f047a817700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0474112a90 con 0x7f0474102af0 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.761+0000 7f047a817700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0474112d90 con 0x7f04741034b0 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0478db4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0474108a80 0x7f04740757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0478db4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0474108a80 0x7f04740757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37066/0 (socket says 172.21.15.72:37066) 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0478db4700 1 -- 172.21.15.72:0/531667783 learned_addr learned my addr 172.21.15.72:0/531667783 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0473fff700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0474102af0 0x7f0474074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:59.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f046b7fe700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04741034b0 0x7f0474075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0478db4700 1 -- 172.21.15.72:0/531667783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04741034b0 msgr2=0x7f0474075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0478db4700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04741034b0 0x7f0474075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0478db4700 1 -- 172.21.15.72:0/531667783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0474102af0 msgr2=0x7f0474074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0478db4700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0474102af0 0x7f0474074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0478db4700 1 -- 172.21.15.72:0/531667783 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04741a7560 con 0x7f0474108a80 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f046b7fe700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04741034b0 0x7f0474075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0478db4700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0474108a80 0x7f04740757c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f046c0062a0 tx=0x7f046c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0471ffb700 1 -- 172.21.15.72:0/531667783 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c021070 con 0x7f0474108a80 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0471ffb700 1 -- 172.21.15.72:0/531667783 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f046c007c80 con 0x7f0474108a80 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f0471ffb700 1 -- 172.21.15.72:0/531667783 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c01d450 con 0x7f0474108a80 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.762+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04741a77f0 con 0x7f0474108a80 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.763+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04741a7ce0 con 0x7f0474108a80 2024-04-25T13:11:59.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.764+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0474069730 con 0x7f0474108a80 2024-04-25T13:11:59.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.765+0000 7f0471ffb700 1 -- 172.21.15.72:0/531667783 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f046c004020 con 0x7f0474108a80 2024-04-25T13:11:59.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.765+0000 7f0471ffb700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04600a3cd0 0x7f04600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:59.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.765+0000 7f0473fff700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04600a3cd0 0x7f04600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:59.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.765+0000 7f0471ffb700 1 -- 172.21.15.72:0/531667783 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f046c00f020 con 0x7f0474108a80 2024-04-25T13:11:59.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.766+0000 7f0473fff700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04600a3cd0 0x7f04600a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f046400abd0 tx=0x7f046400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:59.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.770+0000 7f0471ffb700 1 -- 172.21.15.72:0/531667783 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f046c02a020 con 0x7f0474108a80 2024-04-25T13:11:59.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:11:59 smithi164 ceph-mon[32418]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:59.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.984+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f047402df80 con 0x7f0474108a80 2024-04-25T13:11:59.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.985+0000 7f0471ffb700 1 -- 172.21.15.72:0/531667783 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f046c09e550 con 0x7f0474108a80 2024-04-25T13:11:59.985 INFO:teuthology.orchestra.run.smithi072.stdout:141733920826 2024-04-25T13:11:59.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04600a3cd0 msgr2=0x7f04600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:59.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04600a3cd0 0x7f04600a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f046400abd0 tx=0x7f046400a380 comp rx=0 tx=0).stop 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0474108a80 msgr2=0x7f04740757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0474108a80 0x7f04740757c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f046c0062a0 tx=0x7f046c004e20 comp rx=0 tx=0).stop 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 shutdown_connections 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f04600a3cd0 0x7f04600a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0474108a80 0x7f04740757c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0474102af0 0x7f0474074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 --2- 172.21.15.72:0/531667783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04741034b0 0x7f0474075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 >> 172.21.15.72:0/531667783 conn(0x7f0474079ad0 msgr2=0x7f04741015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 shutdown_connections 2024-04-25T13:11:59.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:11:59.987+0000 7f047a817700 1 -- 172.21.15.72:0/531667783 wait complete. 2024-04-25T13:12:00.688 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:00 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/531667783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:12:00.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:00 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/531667783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:12:00.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:00 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/531667783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:12:00.982 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:01.467 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:01.468 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:01.468 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:01.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:01 smithi072 ceph-mon[29199]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:12:01.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:01 smithi193 ceph-mon[32743]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:12:01.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:01 smithi164 ceph-mon[32418]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:12:01.868 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920826 for osd.4 2024-04-25T13:12:01.869 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:02.265 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574922 for osd.1 2024-04-25T13:12:02.266 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:03.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:02 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1624510627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:12:03.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:02 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1624510627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:12:03.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:02 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1624510627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:12:04.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 -- 172.21.15.72:0/820676074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 msgr2=0x7f6afc107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:04.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/820676074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 0x7f6afc107d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009990 tx=0x7f6aec009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 -- 172.21.15.72:0/820676074 shutdown_connections 2024-04-25T13:12:04.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/820676074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6afc102300 0x7f6afc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/820676074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 0x7f6afc107d70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/820676074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6afc101940 0x7f6afc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 -- 172.21.15.72:0/820676074 >> 172.21.15.72:0/820676074 conn(0x7f6afc0788f0 msgr2=0x7f6afc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 -- 172.21.15.72:0/820676074 shutdown_connections 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 -- 172.21.15.72:0/820676074 wait complete. 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.706+0000 7f6b01e7c700 1 Processor -- start 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6b01e7c700 1 -- start start 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6b01e7c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6afc101940 0x7f6afc19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6b01e7c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6afc102300 0x7f6afc19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6b01e7c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 0x7f6afc19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6b01e7c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc111b60 con 0x7f6afc107990 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6b01e7c700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc1119e0 con 0x7f6afc101940 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6b01e7c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc111ce0 con 0x7f6afc102300 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 0x7f6afc19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:04.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 0x7f6afc19fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37072/0 (socket says 172.21.15.72:37072) 2024-04-25T13:12:04.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afbfff700 1 -- 172.21.15.72:0/1624510627 learned_addr learned my addr 172.21.15.72:0/1624510627 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:04.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afb7fe700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6afc101940 0x7f6afc19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:04.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6af2ffd700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6afc102300 0x7f6afc19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:04.139 INFO:teuthology.orchestra.run.smithi072.stdout:34359738448 2024-04-25T13:12:04.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afbfff700 1 -- 172.21.15.72:0/1624510627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6afc102300 msgr2=0x7f6afc19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:04.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afbfff700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6afc102300 0x7f6afc19b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afbfff700 1 -- 172.21.15.72:0/1624510627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6afc101940 msgr2=0x7f6afc19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:04.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afbfff700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6afc101940 0x7f6afc19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afbfff700 1 -- 172.21.15.72:0/1624510627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc1a0360 con 0x7f6afc107990 2024-04-25T13:12:04.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afb7fe700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6afc101940 0x7f6afc19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:04.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.707+0000 7f6afbfff700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 0x7f6afc19fc20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6af400b970 tx=0x7f6af400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:04.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.708+0000 7f6af97fa700 1 -- 172.21.15.72:0/1624510627 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4019070 con 0x7f6afc107990 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.708+0000 7f6af97fa700 1 -- 172.21.15.72:0/1624510627 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af4014440 con 0x7f6afc107990 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.708+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6afc1a0650 con 0x7f6afc107990 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.708+0000 7f6af97fa700 1 -- 172.21.15.72:0/1624510627 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af400ce60 con 0x7f6afc107990 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.708+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6afc1a7ed0 con 0x7f6afc107990 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.709+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6afc069730 con 0x7f6afc107990 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.710+0000 7f6af97fa700 1 -- 172.21.15.72:0/1624510627 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6af401c070 con 0x7f6afc107990 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.710+0000 7f6af97fa700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6ae80a3cd0 0x7f6ae80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.710+0000 7f6afb7fe700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6ae80a3cd0 0x7f6ae80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.710+0000 7f6af97fa700 1 -- 172.21.15.72:0/1624510627 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f6af40cfbe0 con 0x7f6afc107990 2024-04-25T13:12:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.711+0000 7f6afb7fe700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6ae80a3cd0 0x7f6ae80a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6aec006010 tx=0x7f6aec00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.714+0000 7f6af97fa700 1 -- 172.21.15.72:0/1624510627 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6af409c720 con 0x7f6afc107990 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.921+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6afc1a80a0 con 0x7f6afc107990 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.922+0000 7f6af97fa700 1 -- 172.21.15.72:0/1624510627 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6af409c020 con 0x7f6afc107990 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6ae80a3cd0 msgr2=0x7f6ae80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6ae80a3cd0 0x7f6ae80a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6aec006010 tx=0x7f6aec00b540 comp rx=0 tx=0).stop 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 msgr2=0x7f6afc19fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 0x7f6afc19fc20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6af400b970 tx=0x7f6af400bd30 comp rx=0 tx=0).stop 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 shutdown_connections 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f6ae80a3cd0 0x7f6ae80a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6afc107990 0x7f6afc19fc20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6afc101940 0x7f6afc19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 --2- 172.21.15.72:0/1624510627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6afc102300 0x7f6afc19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 >> 172.21.15.72:0/1624510627 conn(0x7f6afc0788f0 msgr2=0x7f6afc06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:04.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 shutdown_connections 2024-04-25T13:12:04.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:02.924+0000 7f6b01e7c700 1 -- 172.21.15.72:0/1624510627 wait complete. 2024-04-25T13:12:04.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:03 smithi193 ceph-mon[32743]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:04.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:03 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:04.241 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:03 smithi164 ceph-mon[32418]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:04.241 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:03 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:04.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:03 smithi072 ceph-mon[29199]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:04.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:03 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:04.706 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738448 for osd.0 2024-04-25T13:12:04.706 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.153+0000 7f5d30437700 1 -- 172.21.15.72:0/3836792211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d2806bb30 msgr2=0x7f5d2810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.153+0000 7f5d30437700 1 --2- 172.21.15.72:0/3836792211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d2806bb30 0x7f5d2810f900 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b0a0 tx=0x7f5d2000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.153+0000 7f5d30437700 1 -- 172.21.15.72:0/3836792211 shutdown_connections 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.153+0000 7f5d30437700 1 --2- 172.21.15.72:0/3836792211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d2806bb30 0x7f5d2810f900 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.153+0000 7f5d30437700 1 --2- 172.21.15.72:0/3836792211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d28108a30 0x7f5d28108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.153+0000 7f5d30437700 1 --2- 172.21.15.72:0/3836792211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d2806b190 0x7f5d2806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.153+0000 7f5d30437700 1 -- 172.21.15.72:0/3836792211 >> 172.21.15.72:0/3836792211 conn(0x7f5d28078910 msgr2=0x7f5d28078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.153+0000 7f5d30437700 1 -- 172.21.15.72:0/3836792211 shutdown_connections 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.153+0000 7f5d30437700 1 -- 172.21.15.72:0/3836792211 wait complete. 2024-04-25T13:12:05.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d30437700 1 Processor -- start 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d30437700 1 -- start start 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d30437700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d2806b190 0x7f5d28101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d30437700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d2806bb30 0x7f5d28101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d30437700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28108a30 0x7f5d281024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d30437700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5d28112bd0 con 0x7f5d28108a30 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d30437700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d28112a50 con 0x7f5d2806b190 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d30437700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d28112d50 con 0x7f5d2806bb30 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d2e9d4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28108a30 0x7f5d281024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d2e9d4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28108a30 0x7f5d281024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37086/0 (socket says 172.21.15.72:37086) 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d2e9d4700 1 -- 172.21.15.72:0/1109950042 learned_addr learned my addr 172.21.15.72:0/1109950042 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d2e1d3700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d2806b190 0x7f5d28101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.154+0000 7f5d2d9d2700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d2806bb30 0x7f5d28101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:05.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d2e9d4700 1 -- 172.21.15.72:0/1109950042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d2806bb30 msgr2=0x7f5d28101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d2e9d4700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d2806bb30 0x7f5d28101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d2e9d4700 1 -- 172.21.15.72:0/1109950042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d2806b190 msgr2=0x7f5d28101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d2e9d4700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d2806b190 0x7f5d28101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d2e9d4700 1 -- 172.21.15.72:0/1109950042 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d281a7620 con 0x7f5d28108a30 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d2e9d4700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28108a30 0x7f5d281024c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b830 tx=0x7f5d20004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d1f7fe700 1 -- 172.21.15.72:0/1109950042 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20021070 con 0x7f5d28108a30 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d281a78b0 con 0x7f5d28108a30 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d1f7fe700 1 -- 172.21.15.72:0/1109950042 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d20007e30 con 0x7f5d28108a30 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d1f7fe700 1 -- 172.21.15.72:0/1109950042 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20004700 con 0x7f5d28108a30 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.155+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d281a7db0 con 0x7f5d28108a30 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.156+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d28069730 con 0x7f5d28108a30 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.157+0000 7f5d1f7fe700 1 -- 172.21.15.72:0/1109950042 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d20003d70 con 0x7f5d28108a30 2024-04-25T13:12:05.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.158+0000 7f5d1f7fe700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d140a3cd0 0x7f5d140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:05.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.158+0000 7f5d2e1d3700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d140a3cd0 0x7f5d140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:05.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.158+0000 7f5d1f7fe700 1 -- 172.21.15.72:0/1109950042 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5d2000f020 con 0x7f5d28108a30 2024-04-25T13:12:05.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.158+0000 7f5d2e1d3700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d140a3cd0 0x7f5d140a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d180045b0 tx=0x7f5d1800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:05.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.162+0000 7f5d1f7fe700 1 -- 172.21.15.72:0/1109950042 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d2002a020 con 0x7f5d28108a30 2024-04-25T13:12:05.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.376+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5d2802df80 con 0x7f5d28108a30 2024-04-25T13:12:05.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.377+0000 7f5d1f7fe700 1 -- 172.21.15.72:0/1109950042 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5d2009e770 con 0x7f5d28108a30 2024-04-25T13:12:05.377 INFO:teuthology.orchestra.run.smithi072.stdout:356482285593 2024-04-25T13:12:05.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d140a3cd0 msgr2=0x7f5d140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:05.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d140a3cd0 0x7f5d140a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d180045b0 tx=0x7f5d1800b3c0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28108a30 msgr2=0x7f5d281024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:05.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28108a30 0x7f5d281024c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b830 tx=0x7f5d20004ce0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 shutdown_connections 2024-04-25T13:12:05.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f5d140a3cd0 0x7f5d140a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d28108a30 0x7f5d281024c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d2806b190 0x7f5d28101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 --2- 172.21.15.72:0/1109950042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d2806bb30 0x7f5d28101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:05.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 >> 172.21.15.72:0/1109950042 conn(0x7f5d28078910 msgr2=0x7f5d280ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:05.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 shutdown_connections 2024-04-25T13:12:05.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:05.379+0000 7f5d30437700 1 -- 172.21.15.72:0/1109950042 wait complete. 2024-04-25T13:12:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:05 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:05 smithi072 ceph-mon[29199]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:06.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:05 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1109950042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:12:06.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:05 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:05 smithi193 ceph-mon[32743]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:06.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:05 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1109950042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:12:06.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:05 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:06.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:05 smithi164 ceph-mon[32418]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:06.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:05 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1109950042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:12:06.624 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285587 got 356482285593 for osd.11 2024-04-25T13:12:06.624 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:07.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T13:12:07.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]: dispatch 2024-04-25T13:12:07.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-25T13:12:07.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T13:12:07.460 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 7, 6]}]: dispatch 2024-04-25T13:12:07.460 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-25T13:12:07.460 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T13:12:07.460 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-25T13:12:07.460 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:07 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-04-25T13:12:07.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.468+0000 7fae9a959700 1 -- 172.21.15.72:0/961550701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94103450 msgr2=0x7fae9410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:07.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.468+0000 7fae9a959700 1 --2- 172.21.15.72:0/961550701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94103450 0x7fae9410f8b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00b0a0 tx=0x7fae8c00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.468+0000 7fae9a959700 1 -- 172.21.15.72:0/961550701 shutdown_connections 2024-04-25T13:12:07.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.468+0000 7fae9a959700 1 --2- 172.21.15.72:0/961550701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94103450 0x7fae9410f8b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.468+0000 7fae9a959700 1 --2- 172.21.15.72:0/961550701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94108a50 0x7fae94108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.468+0000 7fae9a959700 1 --2- 172.21.15.72:0/961550701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae94102a90 0x7fae94102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.468+0000 7fae9a959700 1 -- 172.21.15.72:0/961550701 >> 172.21.15.72:0/961550701 conn(0x7fae94079ba0 msgr2=0x7fae94079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:07.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.468+0000 7fae9a959700 1 -- 172.21.15.72:0/961550701 shutdown_connections 2024-04-25T13:12:07.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.468+0000 7fae9a959700 1 -- 172.21.15.72:0/961550701 wait complete. 2024-04-25T13:12:07.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae9a959700 1 Processor -- start 2024-04-25T13:12:07.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae9a959700 1 -- start start 2024-04-25T13:12:07.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae9a959700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94102a90 0x7fae94199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:07.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae9a959700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae94103450 0x7fae94199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:07.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae9a959700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94108a50 0x7fae9419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:07.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae9a959700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fae94112ce0 con 0x7fae94108a50 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae9a959700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae94112b60 con 0x7fae94102a90 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae9a959700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae94112e60 con 0x7fae94103450 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae98ef6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94108a50 0x7fae9419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae98ef6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94108a50 0x7fae9419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60542/0 (socket says 172.21.15.72:60542) 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae98ef6700 1 -- 172.21.15.72:0/3461735436 learned_addr learned my addr 172.21.15.72:0/3461735436 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae937fe700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae94103450 0x7fae94199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.469+0000 7fae93fff700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94102a90 0x7fae94199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae98ef6700 1 -- 172.21.15.72:0/3461735436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae94103450 msgr2=0x7fae94199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae98ef6700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae94103450 0x7fae94199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae98ef6700 1 -- 172.21.15.72:0/3461735436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94102a90 msgr2=0x7fae94199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae98ef6700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94102a90 0x7fae94199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae98ef6700 1 -- 172.21.15.72:0/3461735436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9419e240 con 0x7fae94108a50 2024-04-25T13:12:07.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae98ef6700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94108a50 0x7fae9419db00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0062a0 tx=0x7fae8c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:07.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae917fa700 1 -- 172.21.15.72:0/3461735436 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c021070 con 0x7fae94108a50 2024-04-25T13:12:07.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9419e4d0 con 0x7fae94108a50 2024-04-25T13:12:07.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae917fa700 1 -- 172.21.15.72:0/3461735436 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8c003c00 con 0x7fae94108a50 2024-04-25T13:12:07.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae917fa700 1 -- 172.21.15.72:0/3461735436 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c004830 con 0x7fae94108a50 2024-04-25T13:12:07.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.470+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae941a5db0 con 0x7fae94108a50 2024-04-25T13:12:07.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.471+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae94069730 con 0x7fae94108a50 2024-04-25T13:12:07.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.472+0000 7fae917fa700 1 -- 172.21.15.72:0/3461735436 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fae8c007a00 con 0x7fae94108a50 2024-04-25T13:12:07.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.473+0000 7fae917fa700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fae7c0a3cd0 0x7fae7c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:07.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.473+0000 7fae93fff700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fae7c0a3cd0 0x7fae7c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:07.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.473+0000 7fae917fa700 1 -- 172.21.15.72:0/3461735436 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fae8c00f020 con 0x7fae94108a50 2024-04-25T13:12:07.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.473+0000 7fae93fff700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fae7c0a3cd0 0x7fae7c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae84009a10 tx=0x7fae84009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:07.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.477+0000 7fae917fa700 1 -- 172.21.15.72:0/3461735436 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae8c02a020 con 0x7fae94108a50 2024-04-25T13:12:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T13:12:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]: dispatch 2024-04-25T13:12:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-25T13:12:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T13:12:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 7, 6]}]: dispatch 2024-04-25T13:12:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-25T13:12:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T13:12:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-25T13:12:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:07 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-04-25T13:12:07.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T13:12:07.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]: dispatch 2024-04-25T13:12:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-25T13:12:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T13:12:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 7, 6]}]: dispatch 2024-04-25T13:12:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-25T13:12:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T13:12:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-25T13:12:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:07 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-04-25T13:12:07.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.687+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fae9402df80 con 0x7fae94108a50 2024-04-25T13:12:07.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.687+0000 7fae917fa700 1 -- 172.21.15.72:0/3461735436 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fae8c09e470 con 0x7fae94108a50 2024-04-25T13:12:07.687 INFO:teuthology.orchestra.run.smithi072.stdout:223338299437 2024-04-25T13:12:07.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fae7c0a3cd0 msgr2=0x7fae7c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:07.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fae7c0a3cd0 0x7fae7c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae84009a10 tx=0x7fae84009190 comp rx=0 tx=0).stop 2024-04-25T13:12:07.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94108a50 msgr2=0x7fae9419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:07.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94108a50 0x7fae9419db00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0062a0 tx=0x7fae8c004e20 comp rx=0 tx=0).stop 2024-04-25T13:12:07.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 shutdown_connections 2024-04-25T13:12:07.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fae7c0a3cd0 0x7fae7c0a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae94108a50 0x7fae9419db00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94102a90 0x7fae94199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 --2- 172.21.15.72:0/3461735436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae94103450 0x7fae94199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 >> 172.21.15.72:0/3461735436 conn(0x7fae94079ba0 msgr2=0x7fae94101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:07.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 shutdown_connections 2024-04-25T13:12:07.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:07.689+0000 7fae9a959700 1 -- 172.21.15.72:0/3461735436 wait complete. 2024-04-25T13:12:08.458 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]': finished 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 7, 6]}]': finished 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:12:08.459 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:08 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3461735436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:12:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T13:12:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]': finished 2024-04-25T13:12:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-25T13:12:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T13:12:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 7, 6]}]': finished 2024-04-25T13:12:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-25T13:12:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T13:12:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-25T13:12:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-04-25T13:12:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:12:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:12:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:08 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3461735436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:12:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]': finished 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 7, 6]}]': finished 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:12:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:08 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3461735436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:12:09.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.132+0000 7fd75e5e4700 1 -- 172.21.15.72:0/944698935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 msgr2=0x7fd758101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:09.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.132+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/944698935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 0x7fd758101860 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009990 tx=0x7fd74c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.132+0000 7fd75e5e4700 1 -- 172.21.15.72:0/944698935 shutdown_connections 2024-04-25T13:12:09.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.132+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/944698935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758101da0 0x7fd75810f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.132+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/944698935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd758107b00 0x7fd758107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.132+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/944698935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 0x7fd758101860 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.132+0000 7fd75e5e4700 1 -- 172.21.15.72:0/944698935 >> 172.21.15.72:0/944698935 conn(0x7fd758076c60 msgr2=0x7fd758077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.132+0000 7fd75e5e4700 1 -- 172.21.15.72:0/944698935 shutdown_connections 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.132+0000 7fd75e5e4700 1 -- 172.21.15.72:0/944698935 wait complete. 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75e5e4700 1 Processor -- start 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75e5e4700 1 -- start start 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75e5e4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 0x7fd758199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75e5e4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758101da0 0x7fd7581995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75e5e4700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd758107b00 0x7fd75819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75e5e4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd758112c30 con 0x7fd7581013e0 2024-04-25T13:12:09.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75e5e4700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd758112ab0 con 0x7fd758107b00 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75e5e4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd758112db0 con 0x7fd758101da0 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd757fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 0x7fd758199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd757fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 0x7fd758199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60564/0 (socket says 172.21.15.72:60564) 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd757fff700 1 -- 172.21.15.72:0/3440111029 learned_addr learned my addr 172.21.15.72:0/3440111029 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd7577fe700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758101da0 0x7fd7581995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd757fff700 1 -- 172.21.15.72:0/3440111029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758101da0 msgr2=0x7fd7581995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd757fff700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758101da0 0x7fd7581995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd757fff700 1 -- 172.21.15.72:0/3440111029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd758107b00 msgr2=0x7fd75819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75cb81700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd758107b00 0x7fd75819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd757fff700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd758107b00 0x7fd75819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd757fff700 1 -- 172.21.15.72:0/3440111029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75819e090 con 0x7fd7581013e0 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.133+0000 7fd75cb81700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd758107b00 0x7fd75819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.134+0000 7fd757fff700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 0x7fd758199080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd74800c990 tx=0x7fd74800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:09.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.134+0000 7fd7557fa700 1 -- 172.21.15.72:0/3440111029 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748007b00 con 0x7fd7581013e0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.134+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75819e380 con 0x7fd7581013e0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.134+0000 7fd7557fa700 1 -- 172.21.15.72:0/3440111029 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd748007ca0 con 0x7fd7581013e0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.134+0000 7fd7557fa700 1 -- 172.21.15.72:0/3440111029 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748018660 con 0x7fd7581013e0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.134+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7581a5c00 con 0x7fd7581013e0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.135+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd758069730 con 0x7fd7581013e0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.137+0000 7fd7557fa700 1 -- 172.21.15.72:0/3440111029 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd748004030 con 0x7fd7581013e0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.137+0000 7fd7557fa700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd7400a3cd0 0x7fd7400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.137+0000 7fd7577fe700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd7400a3cd0 0x7fd7400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.137+0000 7fd7557fa700 1 -- 172.21.15.72:0/3440111029 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fd7480d0640 con 0x7fd7581013e0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.137+0000 7fd7577fe700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd7400a3cd0 0x7fd7400a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd74c006010 tx=0x7fd74c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:09.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.141+0000 7fd7557fa700 1 -- 172.21.15.72:0/3440111029 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd748014030 con 0x7fd7581013e0 2024-04-25T13:12:09.284 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:09 smithi072 ceph-mon[29199]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:12:09.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.350+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd75802df80 con 0x7fd7581013e0 2024-04-25T13:12:09.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.350+0000 7fd7557fa700 1 -- 172.21.15.72:0/3440111029 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd74809cf40 con 0x7fd7581013e0 2024-04-25T13:12:09.351 INFO:teuthology.orchestra.run.smithi072.stdout:201863462964 2024-04-25T13:12:09.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd7400a3cd0 msgr2=0x7fd7400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:09.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd7400a3cd0 0x7fd7400a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd74c006010 tx=0x7fd74c01a040 comp rx=0 tx=0).stop 2024-04-25T13:12:09.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 msgr2=0x7fd758199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:09.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 0x7fd758199080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd74800c990 tx=0x7fd74800cd50 comp rx=0 tx=0).stop 2024-04-25T13:12:09.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 shutdown_connections 2024-04-25T13:12:09.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd758107b00 0x7fd75819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd7400a3cd0 0x7fd7400a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7581013e0 0x7fd758199080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 --2- 172.21.15.72:0/3440111029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758101da0 0x7fd7581995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:09.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 >> 172.21.15.72:0/3440111029 conn(0x7fd758076c60 msgr2=0x7fd7580fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:09.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 shutdown_connections 2024-04-25T13:12:09.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:09.352+0000 7fd75e5e4700 1 -- 172.21.15.72:0/3440111029 wait complete. 2024-04-25T13:12:09.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:09 smithi193 ceph-mon[32743]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:12:09.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:09 smithi164 ceph-mon[32418]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:12:09.984 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462964 for osd.6 2024-04-25T13:12:09.985 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:10.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 -- 172.21.15.72:0/2368678348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 msgr2=0x7fb02010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2368678348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 0x7fb02010e7f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb01800b0a0 tx=0x7fb01800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 -- 172.21.15.72:0/2368678348 shutdown_connections 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2368678348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 0x7fb02010e7f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2368678348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb020107990 0x7fb020107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2368678348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb020101940 0x7fb020101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 -- 172.21.15.72:0/2368678348 >> 172.21.15.72:0/2368678348 conn(0x7fb0200788f0 msgr2=0x7fb020078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 -- 172.21.15.72:0/2368678348 shutdown_connections 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 -- 172.21.15.72:0/2368678348 wait complete. 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.143+0000 7fb0257cb700 1 Processor -- start 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb0257cb700 1 -- start start 2024-04-25T13:12:10.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb0257cb700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb020101940 0x7fb0201034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb0257cb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 0x7fb020103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb0257cb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb020107990 0x7fb020103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb0257cb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb020111af0 con 0x7fb020102300 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb0257cb700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb020111970 con 0x7fb020101940 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb0257cb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb020111c70 con 0x7fb020107990 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01e7fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 0x7fb020103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01e7fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 0x7fb020103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60578/0 (socket says 172.21.15.72:60578) 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01e7fc700 1 -- 172.21.15.72:0/2158823437 learned_addr learned my addr 172.21.15.72:0/2158823437 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01effd700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb020101940 0x7fb0201034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01e7fc700 1 -- 172.21.15.72:0/2158823437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb020107990 msgr2=0x7fb020103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01e7fc700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb020107990 0x7fb020103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01e7fc700 1 -- 172.21.15.72:0/2158823437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb020101940 msgr2=0x7fb0201034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:10.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01e7fc700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb020101940 0x7fb0201034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01e7fc700 1 -- 172.21.15.72:0/2158823437 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0201a75c0 con 0x7fb020102300 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb01e7fc700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 0x7fb020103a30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d980 tx=0x7fb01400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb007fff700 1 -- 172.21.15.72:0/2158823437 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014017070 con 0x7fb020102300 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb007fff700 1 -- 172.21.15.72:0/2158823437 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb014004830 con 0x7fb020102300 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.144+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0201a78b0 con 0x7fb020102300 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.145+0000 7fb007fff700 1 -- 172.21.15.72:0/2158823437 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014005440 con 0x7fb020102300 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.145+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0201a7da0 con 0x7fb020102300 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.145+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb020069730 con 0x7fb020102300 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.146+0000 7fb007fff700 1 -- 172.21.15.72:0/2158823437 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb0140055e0 con 0x7fb020102300 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.147+0000 7fb007fff700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0080a3cd0 0x7fb0080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.147+0000 7fb01effd700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0080a3cd0 0x7fb0080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.147+0000 7fb007fff700 1 -- 172.21.15.72:0/2158823437 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fb0140d08f0 con 0x7fb020102300 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.147+0000 7fb01effd700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0080a3cd0 0x7fb0080a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb010005fd0 tx=0x7fb010005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:10.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.151+0000 7fb007fff700 1 -- 172.21.15.72:0/2158823437 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb014020020 con 0x7fb020102300 2024-04-25T13:12:10.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:10 smithi072 ceph-mon[29199]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T13:12:10.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:10 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3440111029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:12:10.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.358+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb02002e140 con 0x7fb020102300 2024-04-25T13:12:10.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.358+0000 7fb007fff700 1 -- 172.21.15.72:0/2158823437 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb01409d160 con 0x7fb020102300 2024-04-25T13:12:10.358 INFO:teuthology.orchestra.run.smithi072.stdout:261993005097 2024-04-25T13:12:10.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0080a3cd0 msgr2=0x7fb0080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:10.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0080a3cd0 0x7fb0080a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb010005fd0 tx=0x7fb010005f60 comp rx=0 tx=0).stop 2024-04-25T13:12:10.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 msgr2=0x7fb020103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:10.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 0x7fb020103a30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d980 tx=0x7fb01400dd40 comp rx=0 tx=0).stop 2024-04-25T13:12:10.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 shutdown_connections 2024-04-25T13:12:10.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb020107990 0x7fb020103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb020101940 0x7fb0201034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fb0080a3cd0 0x7fb0080a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 --2- 172.21.15.72:0/2158823437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb020102300 0x7fb020103a30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:10.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 >> 172.21.15.72:0/2158823437 conn(0x7fb0200788f0 msgr2=0x7fb02006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:10.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 shutdown_connections 2024-04-25T13:12:10.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:10.360+0000 7fb0257cb700 1 -- 172.21.15.72:0/2158823437 wait complete. 2024-04-25T13:12:10.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:10 smithi193 ceph-mon[32743]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T13:12:10.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:10 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3440111029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:12:10.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:10 smithi164 ceph-mon[32418]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T13:12:10.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:10 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3440111029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:12:11.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:11 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2158823437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:12:11.260 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005091 got 261993005097 for osd.8 2024-04-25T13:12:11.261 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:11.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:11 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2158823437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:12:11.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:11 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2158823437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:12:12.239 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:12 smithi072 ceph-mon[29199]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:12:12.265 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299433 got 223338299437 for osd.7 2024-04-25T13:12:12.265 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:12.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:12 smithi193 ceph-mon[32743]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:12:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:12 smithi164 ceph-mon[32418]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:12:13.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 -- 172.21.15.72:0/3562612064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c102ac0 msgr2=0x7fe12c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:13.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 --2- 172.21.15.72:0/3562612064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c102ac0 0x7fe12c102f40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe120009990 tx=0x7fe120009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 -- 172.21.15.72:0/3562612064 shutdown_connections 2024-04-25T13:12:13.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 --2- 172.21.15.72:0/3562612064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12c103480 0x7fe12c10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 --2- 172.21.15.72:0/3562612064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe12c108a70 0x7fe12c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 --2- 172.21.15.72:0/3562612064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c102ac0 0x7fe12c102f40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 -- 172.21.15.72:0/3562612064 >> 172.21.15.72:0/3562612064 conn(0x7fe12c079bb0 msgr2=0x7fe12c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 -- 172.21.15.72:0/3562612064 shutdown_connections 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 -- 172.21.15.72:0/3562612064 wait complete. 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 Processor -- start 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.410+0000 7fe132274700 1 -- start start 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe132274700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe12c102ac0 0x7fe12c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe132274700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12c103480 0x7fe12c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe132274700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c108a70 0x7fe12c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe132274700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c112c80 con 0x7fe12c108a70 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe132274700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c112b00 con 0x7fe12c102ac0 2024-04-25T13:12:13.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe132274700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c112e00 con 0x7fe12c103480 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe130811700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c108a70 0x7fe12c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe130811700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c108a70 0x7fe12c19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60604/0 (socket says 172.21.15.72:60604) 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe130811700 1 -- 172.21.15.72:0/349983562 learned_addr learned my addr 172.21.15.72:0/349983562 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe12bfff700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe12c102ac0 0x7fe12c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe12b7fe700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12c103480 0x7fe12c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe130811700 1 -- 172.21.15.72:0/349983562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12c103480 msgr2=0x7fe12c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe130811700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12c103480 0x7fe12c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe130811700 1 -- 172.21.15.72:0/349983562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe12c102ac0 msgr2=0x7fe12c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe130811700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe12c102ac0 0x7fe12c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.411+0000 7fe130811700 1 -- 172.21.15.72:0/349983562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c19e0e0 con 0x7fe12c108a70 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.412+0000 7fe130811700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c108a70 0x7fe12c19d9a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe124007d70 tx=0x7fe12400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:13.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.412+0000 7fe1297fa700 1 -- 172.21.15.72:0/349983562 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124016020 con 0x7fe12c108a70 2024-04-25T13:12:13.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.412+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c19e3d0 con 0x7fe12c108a70 2024-04-25T13:12:13.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.412+0000 7fe1297fa700 1 -- 172.21.15.72:0/349983562 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe12400ad40 con 0x7fe12c108a70 2024-04-25T13:12:13.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.412+0000 7fe1297fa700 1 -- 172.21.15.72:0/349983562 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1240115e0 con 0x7fe12c108a70 2024-04-25T13:12:13.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.412+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c1a5c50 con 0x7fe12c108a70 2024-04-25T13:12:13.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.413+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12c069730 con 0x7fe12c108a70 2024-04-25T13:12:13.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.414+0000 7fe1297fa700 1 -- 172.21.15.72:0/349983562 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe124019020 con 0x7fe12c108a70 2024-04-25T13:12:13.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.414+0000 7fe1297fa700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1140a3cd0 0x7fe1140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:13.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.414+0000 7fe12bfff700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1140a3cd0 0x7fe1140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:13.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.415+0000 7fe1297fa700 1 -- 172.21.15.72:0/349983562 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fe1240d04b0 con 0x7fe12c108a70 2024-04-25T13:12:13.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.415+0000 7fe12bfff700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1140a3cd0 0x7fe1140a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe11c006fd0 tx=0x7fe11c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:13.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.419+0000 7fe1297fa700 1 -- 172.21.15.72:0/349983562 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe124020020 con 0x7fe12c108a70 2024-04-25T13:12:13.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.620+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe12c02df80 con 0x7fe12c108a70 2024-04-25T13:12:13.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.620+0000 7fe1297fa700 1 -- 172.21.15.72:0/349983562 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe12409ce30 con 0x7fe12c108a70 2024-04-25T13:12:13.621 INFO:teuthology.orchestra.run.smithi072.stdout:167503724602 2024-04-25T13:12:13.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1140a3cd0 msgr2=0x7fe1140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:13.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1140a3cd0 0x7fe1140a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe11c006fd0 tx=0x7fe11c008040 comp rx=0 tx=0).stop 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c108a70 msgr2=0x7fe12c19d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c108a70 0x7fe12c19d9a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe124007d70 tx=0x7fe12400a340 comp rx=0 tx=0).stop 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 shutdown_connections 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe1140a3cd0 0x7fe1140a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe12c108a70 0x7fe12c19d9a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe12c102ac0 0x7fe12c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 --2- 172.21.15.72:0/349983562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12c103480 0x7fe12c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.622+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 >> 172.21.15.72:0/349983562 conn(0x7fe12c079bb0 msgr2=0x7fe12c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.623+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 shutdown_connections 2024-04-25T13:12:13.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:13.623+0000 7fe132274700 1 -- 172.21.15.72:0/349983562 wait complete. 2024-04-25T13:12:14.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:14 smithi072 ceph-mon[29199]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:12:14.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:14 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/349983562' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:12:14.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724602 for osd.5 2024-04-25T13:12:14.321 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:14.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:14 smithi193 ceph-mon[32743]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:12:14.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:14 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/349983562' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:12:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.529+0000 7f1b70fd3700 1 -- 172.21.15.72:0/932707633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c103450 msgr2=0x7f1b6c10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.529+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/932707633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c103450 0x7f1b6c10f8b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400b0a0 tx=0x7f1b6400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.529+0000 7f1b70fd3700 1 -- 172.21.15.72:0/932707633 shutdown_connections 2024-04-25T13:12:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.529+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/932707633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c103450 0x7f1b6c10f8b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.529+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/932707633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c108a50 0x7f1b6c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.529+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/932707633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b6c102a90 0x7f1b6c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.529+0000 7f1b70fd3700 1 -- 172.21.15.72:0/932707633 >> 172.21.15.72:0/932707633 conn(0x7f1b6c079ba0 msgr2=0x7f1b6c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.529+0000 7f1b70fd3700 1 -- 172.21.15.72:0/932707633 shutdown_connections 2024-04-25T13:12:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.529+0000 7f1b70fd3700 1 -- 172.21.15.72:0/932707633 wait complete. 2024-04-25T13:12:14.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.530+0000 7f1b70fd3700 1 Processor -- start 2024-04-25T13:12:14.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.530+0000 7f1b70fd3700 1 -- start start 2024-04-25T13:12:14.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.530+0000 7f1b70fd3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b6c102a90 0x7f1b6c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.530+0000 7f1b70fd3700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103450 0x7f1b6c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.530+0000 7f1b70fd3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c108a50 0x7f1b6c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.530+0000 7f1b70fd3700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c112be0 con 0x7f1b6c108a50 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.530+0000 7f1b70fd3700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c112a60 con 0x7f1b6c103450 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b70fd3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c112d60 con 0x7f1b6c102a90 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c108a50 0x7f1b6c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c108a50 0x7f1b6c0757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60626/0 (socket says 172.21.15.72:60626) 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b69d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103450 0x7f1b6c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6ad9d700 1 -- 172.21.15.72:0/2223764766 learned_addr learned my addr 172.21.15.72:0/2223764766 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6a59c700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b6c102a90 0x7f1b6c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6ad9d700 1 -- 172.21.15.72:0/2223764766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b6c102a90 msgr2=0x7f1b6c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6ad9d700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b6c102a90 0x7f1b6c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6ad9d700 1 -- 172.21.15.72:0/2223764766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103450 msgr2=0x7f1b6c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6ad9d700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103450 0x7f1b6c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6ad9d700 1 -- 172.21.15.72:0/2223764766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c1a76a0 con 0x7f1b6c108a50 2024-04-25T13:12:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b6ad9d700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c108a50 0x7f1b6c0757c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400b830 tx=0x7f1b64004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b5b7fe700 1 -- 172.21.15.72:0/2223764766 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64021070 con 0x7f1b6c108a50 2024-04-25T13:12:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c1a7930 con 0x7f1b6c108a50 2024-04-25T13:12:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b5b7fe700 1 -- 172.21.15.72:0/2223764766 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b64007e30 con 0x7f1b6c108a50 2024-04-25T13:12:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b5b7fe700 1 -- 172.21.15.72:0/2223764766 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64004700 con 0x7f1b6c108a50 2024-04-25T13:12:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.531+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c1a7ec0 con 0x7f1b6c108a50 2024-04-25T13:12:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.532+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c069730 con 0x7f1b6c108a50 2024-04-25T13:12:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.533+0000 7f1b5b7fe700 1 -- 172.21.15.72:0/2223764766 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b64003d70 con 0x7f1b6c108a50 2024-04-25T13:12:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.534+0000 7f1b5b7fe700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1b540a3cd0 0x7f1b540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.534+0000 7f1b6a59c700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1b540a3cd0 0x7f1b540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.534+0000 7f1b5b7fe700 1 -- 172.21.15.72:0/2223764766 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1b6400f020 con 0x7f1b6c108a50 2024-04-25T13:12:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.534+0000 7f1b6a59c700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1b540a3cd0 0x7f1b540a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c005950 tx=0x7f1b5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:14.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.538+0000 7f1b5b7fe700 1 -- 172.21.15.72:0/2223764766 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b6402a020 con 0x7f1b6c108a50 2024-04-25T13:12:14.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:14 smithi164 ceph-mon[32418]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:12:14.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:14 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/349983562' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:12:14.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.739+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f1b6c02df80 con 0x7f1b6c108a50 2024-04-25T13:12:14.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.739+0000 7f1b5b7fe700 1 -- 172.21.15.72:0/2223764766 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1b6409e700 con 0x7f1b6c108a50 2024-04-25T13:12:14.740 INFO:teuthology.orchestra.run.smithi072.stdout:296352743460 2024-04-25T13:12:14.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1b540a3cd0 msgr2=0x7f1b540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:14.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1b540a3cd0 0x7f1b540a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c005950 tx=0x7f1b5c00b410 comp rx=0 tx=0).stop 2024-04-25T13:12:14.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c108a50 msgr2=0x7f1b6c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c108a50 0x7f1b6c0757c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400b830 tx=0x7f1b64004ce0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 shutdown_connections 2024-04-25T13:12:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1b540a3cd0 0x7f1b540a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b6c108a50 0x7f1b6c0757c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b6c102a90 0x7f1b6c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 --2- 172.21.15.72:0/2223764766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103450 0x7f1b6c075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 >> 172.21.15.72:0/2223764766 conn(0x7f1b6c079ba0 msgr2=0x7f1b6c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 shutdown_connections 2024-04-25T13:12:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:14.741+0000 7f1b70fd3700 1 -- 172.21.15.72:0/2223764766 wait complete. 2024-04-25T13:12:15.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:15 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2223764766' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:12:15.504 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:15 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2223764766' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:12:15.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:15 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2223764766' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:12:15.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.607+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3158789340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4103760 msgr2=0x7f2df4103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:15.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.607+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3158789340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4103760 0x7f2df4103b40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009990 tx=0x7f2df0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.607+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3158789340 shutdown_connections 2024-04-25T13:12:15.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.607+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3158789340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df4102330 0x7f2df410e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.607+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3158789340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4103760 0x7f2df4103b40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.607+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3158789340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df4101970 0x7f2df4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.607+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3158789340 >> 172.21.15.72:0/3158789340 conn(0x7f2df40779b0 msgr2=0x7f2df4077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:15.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.607+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3158789340 shutdown_connections 2024-04-25T13:12:15.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.607+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3158789340 wait complete. 2024-04-25T13:12:15.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2dfbd88700 1 Processor -- start 2024-04-25T13:12:15.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2dfbd88700 1 -- start start 2024-04-25T13:12:15.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2dfbd88700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df4101970 0x7f2df419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:15.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2dfbd88700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4102330 0x7f2df419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2dfbd88700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df4103760 0x7f2df41a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2dfbd88700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2df4111cc0 con 0x7f2df4102330 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2dfbd88700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2df4111b40 con 0x7f2df4103760 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2dfbd88700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2df4111e40 con 0x7f2df4101970 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2df9323700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4102330 0x7f2df419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2df9323700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4102330 0x7f2df419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43024/0 (socket says 172.21.15.72:43024) 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.608+0000 7f2df9323700 1 -- 172.21.15.72:0/3619106555 learned_addr learned my addr 172.21.15.72:0/3619106555 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2df9b24700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df4101970 0x7f2df419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2dfa325700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df4103760 0x7f2df41a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2df9323700 1 -- 172.21.15.72:0/3619106555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df4101970 msgr2=0x7f2df419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2df9323700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df4101970 0x7f2df419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2df9323700 1 -- 172.21.15.72:0/3619106555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df4103760 msgr2=0x7f2df41a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:15.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2df9323700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df4103760 0x7f2df41a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2df9323700 1 -- 172.21.15.72:0/3619106555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df41a5760 con 0x7f2df4102330 2024-04-25T13:12:15.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2df9b24700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df4101970 0x7f2df419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:15.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2df9323700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4102330 0x7f2df419a6e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b530 tx=0x7f2de400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:15.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2deaffd700 1 -- 172.21.15.72:0/3619106555 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4015020 con 0x7f2df4102330 2024-04-25T13:12:15.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2deaffd700 1 -- 172.21.15.72:0/3619106555 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de4011ce0 con 0x7f2df4102330 2024-04-25T13:12:15.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df41a5990 con 0x7f2df4102330 2024-04-25T13:12:15.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2deaffd700 1 -- 172.21.15.72:0/3619106555 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4010e40 con 0x7f2df4102330 2024-04-25T13:12:15.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.609+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df41a5ea0 con 0x7f2df4102330 2024-04-25T13:12:15.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.610+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df410a020 con 0x7f2df4102330 2024-04-25T13:12:15.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.612+0000 7f2deaffd700 1 -- 172.21.15.72:0/3619106555 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2de4018020 con 0x7f2df4102330 2024-04-25T13:12:15.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.613+0000 7f2deaffd700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f2de00a3cd0 0x7f2de00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:15.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.613+0000 7f2df9b24700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f2de00a3cd0 0x7f2de00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:15.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.613+0000 7f2deaffd700 1 -- 172.21.15.72:0/3619106555 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f2de40d01a0 con 0x7f2df4102330 2024-04-25T13:12:15.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.613+0000 7f2df9b24700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f2de00a3cd0 0x7f2de00a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006010 tx=0x7f2df000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:15.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.616+0000 7f2deaffd700 1 -- 172.21.15.72:0/3619106555 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2de401f020 con 0x7f2df4102330 2024-04-25T13:12:15.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.824+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f2df406b7e0 con 0x7f2df4102330 2024-04-25T13:12:15.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.825+0000 7f2deaffd700 1 -- 172.21.15.72:0/3619106555 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2de409cb20 con 0x7f2df4102330 2024-04-25T13:12:15.825 INFO:teuthology.orchestra.run.smithi072.stdout:322122547233 2024-04-25T13:12:15.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f2de00a3cd0 msgr2=0x7f2de00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:15.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f2de00a3cd0 0x7f2de00a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006010 tx=0x7f2df000b540 comp rx=0 tx=0).stop 2024-04-25T13:12:15.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4102330 msgr2=0x7f2df419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:15.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4102330 0x7f2df419a6e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b530 tx=0x7f2de400b840 comp rx=0 tx=0).stop 2024-04-25T13:12:15.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 shutdown_connections 2024-04-25T13:12:15.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df4103760 0x7f2df41a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df4101970 0x7f2df419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f2de00a3cd0 0x7f2de00a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 --2- 172.21.15.72:0/3619106555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2df4102330 0x7f2df419a6e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:15.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 >> 172.21.15.72:0/3619106555 conn(0x7f2df40779b0 msgr2=0x7f2df4106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:15.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 shutdown_connections 2024-04-25T13:12:15.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:15.827+0000 7f2dfbd88700 1 -- 172.21.15.72:0/3619106555 wait complete. 2024-04-25T13:12:16.089 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743453 got 296352743460 for osd.9 2024-04-25T13:12:16.089 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:16.376 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:16 smithi072 ceph-mon[29199]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:12:16.376 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:16 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3619106555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:12:16.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:16 smithi193 ceph-mon[32743]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:12:16.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:16 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3619106555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:12:16.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:16 smithi164 ceph-mon[32418]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:12:16.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:16 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3619106555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:12:16.928 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547227 got 322122547233 for osd.10 2024-04-25T13:12:16.929 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:17.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.164+0000 7ffba4f1a700 1 -- 172.21.15.72:0/3984586769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba0101930 msgr2=0x7ffba0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:17.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.164+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/3984586769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba0101930 0x7ffba0101db0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009990 tx=0x7ffb94009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.164+0000 7ffba4f1a700 1 -- 172.21.15.72:0/3984586769 shutdown_connections 2024-04-25T13:12:17.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.164+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/3984586769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba01022f0 0x7ffba010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.164+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/3984586769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffba0107980 0x7ffba0107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.164+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/3984586769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba0101930 0x7ffba0101db0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.164+0000 7ffba4f1a700 1 -- 172.21.15.72:0/3984586769 >> 172.21.15.72:0/3984586769 conn(0x7ffba00788b0 msgr2=0x7ffba0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:17.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.164+0000 7ffba4f1a700 1 -- 172.21.15.72:0/3984586769 shutdown_connections 2024-04-25T13:12:17.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.164+0000 7ffba4f1a700 1 -- 172.21.15.72:0/3984586769 wait complete. 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffba4f1a700 1 Processor -- start 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffba4f1a700 1 -- start start 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffba4f1a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffba0101930 0x7ffba0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffba4f1a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba01022f0 0x7ffba019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffba4f1a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0107980 0x7ffba01a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffba4f1a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0111b90 con 0x7ffba01022f0 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffba4f1a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0111a10 con 0x7ffba0101930 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffba4f1a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0111d10 con 0x7ffba0107980 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffb9dd9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba01022f0 0x7ffba019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffb9dd9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba01022f0 0x7ffba019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43044/0 (socket says 172.21.15.72:43044) 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffb9dd9b700 1 -- 172.21.15.72:0/532751636 learned_addr learned my addr 172.21.15.72:0/532751636 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffb9ed9d700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0107980 0x7ffba01a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:17.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.165+0000 7ffb9e59c700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffba0101930 0x7ffba0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffb9dd9b700 1 -- 172.21.15.72:0/532751636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0107980 msgr2=0x7ffba01a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffb9dd9b700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0107980 0x7ffba01a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffb9dd9b700 1 -- 172.21.15.72:0/532751636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffba0101930 msgr2=0x7ffba0199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffb9dd9b700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffba0101930 0x7ffba0199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffb9dd9b700 1 -- 172.21.15.72:0/532751636 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba01a5530 con 0x7ffba01022f0 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffb9dd9b700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba01022f0 0x7ffba019a4b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffb94005910 tx=0x7ffb940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffb8f7fe700 1 -- 172.21.15.72:0/532751636 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94027070 con 0x7ffba01022f0 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba0197d60 con 0x7ffba01022f0 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffb8f7fe700 1 -- 172.21.15.72:0/532751636 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb94004020 con 0x7ffba01022f0 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffb8f7fe700 1 -- 172.21.15.72:0/532751636 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94022e00 con 0x7ffba01022f0 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.166+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba0198210 con 0x7ffba01022f0 2024-04-25T13:12:17.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.167+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba0069730 con 0x7ffba01022f0 2024-04-25T13:12:17.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.168+0000 7ffb8f7fe700 1 -- 172.21.15.72:0/532751636 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffb9402a070 con 0x7ffba01022f0 2024-04-25T13:12:17.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.168+0000 7ffb8f7fe700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb880a3cd0 0x7ffb880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:17.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.169+0000 7ffb9e59c700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb880a3cd0 0x7ffb880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:17.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.169+0000 7ffb8f7fe700 1 -- 172.21.15.72:0/532751636 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ffb940d1460 con 0x7ffba01022f0 2024-04-25T13:12:17.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.169+0000 7ffb9e59c700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb880a3cd0 0x7ffb880a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb90005fd0 tx=0x7ffb9000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:17.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.174+0000 7ffb8f7fe700 1 -- 172.21.15.72:0/532751636 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb9405eff0 con 0x7ffba01022f0 2024-04-25T13:12:17.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.381+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ffba002df80 con 0x7ffba01022f0 2024-04-25T13:12:17.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.381+0000 7ffb8f7fe700 1 -- 172.21.15.72:0/532751636 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffb94013070 con 0x7ffba01022f0 2024-04-25T13:12:17.382 INFO:teuthology.orchestra.run.smithi072.stdout:107374182467 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb880a3cd0 msgr2=0x7ffb880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb880a3cd0 0x7ffb880a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb90005fd0 tx=0x7ffb9000a560 comp rx=0 tx=0).stop 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba01022f0 msgr2=0x7ffba019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba01022f0 0x7ffba019a4b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffb94005910 tx=0x7ffb940047b0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 shutdown_connections 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0107980 0x7ffba01a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffba0101930 0x7ffba0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffb880a3cd0 0x7ffb880a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 --2- 172.21.15.72:0/532751636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffba01022f0 0x7ffba019a4b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 >> 172.21.15.72:0/532751636 conn(0x7ffba00788b0 msgr2=0x7ffba00ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 shutdown_connections 2024-04-25T13:12:17.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:17.384+0000 7ffba4f1a700 1 -- 172.21.15.72:0/532751636 wait complete. 2024-04-25T13:12:18.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:18 smithi193 ceph-mon[32743]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 18 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:12:18.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:18 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/532751636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:12:18.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:18 smithi072 ceph-mon[29199]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 18 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:12:18.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:18 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/532751636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:12:18.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:18 smithi164 ceph-mon[32418]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 18 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:12:18.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:18 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/532751636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:12:18.593 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182467 for osd.3 2024-04-25T13:12:18.593 DEBUG:teuthology.parallel:result is None 2024-04-25T13:12:18.594 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T13:12:18.594 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:12:18.858 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:20.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:20 smithi072 ceph-mon[29199]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:12:20.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:20.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:20 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:20.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:20 smithi193 ceph-mon[32743]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:12:20.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:20 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:20.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:20 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:20.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:20 smithi164 ceph-mon[32418]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:12:20.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:20 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:20.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:20 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:20.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.590+0000 7f3bf929a700 1 -- 172.21.15.72:0/3816737684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 msgr2=0x7f3bf4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:20.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.590+0000 7f3bf929a700 1 --2- 172.21.15.72:0/3816737684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 0x7f3bf4102f10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009990 tx=0x7f3be4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.590+0000 7f3bf929a700 1 -- 172.21.15.72:0/3816737684 shutdown_connections 2024-04-25T13:12:20.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.590+0000 7f3bf929a700 1 --2- 172.21.15.72:0/3816737684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bf4103450 0x7f3bf410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.590+0000 7f3bf929a700 1 --2- 172.21.15.72:0/3816737684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bf4108a50 0x7f3bf4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.590+0000 7f3bf929a700 1 --2- 172.21.15.72:0/3816737684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 0x7f3bf4102f10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.590+0000 7f3bf929a700 1 -- 172.21.15.72:0/3816737684 >> 172.21.15.72:0/3816737684 conn(0x7f3bf4079ba0 msgr2=0x7f3bf4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:20.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.590+0000 7f3bf929a700 1 -- 172.21.15.72:0/3816737684 shutdown_connections 2024-04-25T13:12:20.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.590+0000 7f3bf929a700 1 -- 172.21.15.72:0/3816737684 wait complete. 2024-04-25T13:12:20.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.591+0000 7f3bf929a700 1 Processor -- start 2024-04-25T13:12:20.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.591+0000 7f3bf929a700 1 -- start start 2024-04-25T13:12:20.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.591+0000 7f3bf929a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 0x7f3bf41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:20.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.591+0000 7f3bf929a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bf4103450 0x7f3bf4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:20.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.591+0000 7f3bf929a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bf4108a50 0x7f3bf419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.591+0000 7f3bf929a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf4112ce0 con 0x7f3bf4102a90 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.591+0000 7f3bf929a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf4112b60 con 0x7f3bf4103450 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.591+0000 7f3bf929a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf4112e60 con 0x7f3bf4108a50 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf2ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 0x7f3bf41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf2ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 0x7f3bf41992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43060/0 (socket says 172.21.15.72:43060) 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf2ffd700 1 -- 172.21.15.72:0/1082178284 learned_addr learned my addr 172.21.15.72:0/1082178284 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf27fc700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bf4103450 0x7f3bf4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf37fe700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bf4108a50 0x7f3bf419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf2ffd700 1 -- 172.21.15.72:0/1082178284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bf4108a50 msgr2=0x7f3bf419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf2ffd700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bf4108a50 0x7f3bf419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf2ffd700 1 -- 172.21.15.72:0/1082178284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bf4103450 msgr2=0x7f3bf4199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:20.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf2ffd700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bf4103450 0x7f3bf4199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf2ffd700 1 -- 172.21.15.72:0/1082178284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf419e2e0 con 0x7f3bf4102a90 2024-04-25T13:12:20.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf27fc700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bf4103450 0x7f3bf4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:20.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf2ffd700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 0x7f3bf41992d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3be800b530 tx=0x7f3be800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:20.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bdbfff700 1 -- 172.21.15.72:0/1082178284 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8015020 con 0x7f3bf4102a90 2024-04-25T13:12:20.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf419e5d0 con 0x7f3bf4102a90 2024-04-25T13:12:20.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bdbfff700 1 -- 172.21.15.72:0/1082178284 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be8011ce0 con 0x7f3bf4102a90 2024-04-25T13:12:20.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bdbfff700 1 -- 172.21.15.72:0/1082178284 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8010d20 con 0x7f3bf4102a90 2024-04-25T13:12:20.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.592+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf41a5e50 con 0x7f3bf4102a90 2024-04-25T13:12:20.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.593+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf4069730 con 0x7f3bf4102a90 2024-04-25T13:12:20.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.595+0000 7f3bdbfff700 1 -- 172.21.15.72:0/1082178284 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3be8018020 con 0x7f3bf4102a90 2024-04-25T13:12:20.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.595+0000 7f3bdbfff700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3bdc0a3cd0 0x7f3bdc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:20.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.595+0000 7f3bf27fc700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3bdc0a3cd0 0x7f3bdc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:20.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.596+0000 7f3bdbfff700 1 -- 172.21.15.72:0/1082178284 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f3be80d0210 con 0x7f3bf4102a90 2024-04-25T13:12:20.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.596+0000 7f3bf27fc700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3bdc0a3cd0 0x7f3bdc0a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3be4006010 tx=0x7f3be401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:20.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.599+0000 7f3bdbfff700 1 -- 172.21.15.72:0/1082178284 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3be801f020 con 0x7f3bf4102a90 2024-04-25T13:12:20.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.793+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3bf419a630 con 0x7f3bdc0a3cd0 2024-04-25T13:12:20.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.801+0000 7f3bdbfff700 1 -- 172.21.15.72:0/1082178284 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258727 (secure 0 0 0) 0x7f3bf419a630 con 0x7f3bdc0a3cd0 2024-04-25T13:12:20.804 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:20.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3bdc0a3cd0 msgr2=0x7f3bdc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:20.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3bdc0a3cd0 0x7f3bdc0a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3be4006010 tx=0x7f3be401a040 comp rx=0 tx=0).stop 2024-04-25T13:12:20.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 msgr2=0x7f3bf41992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:20.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 0x7f3bf41992d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3be800b530 tx=0x7f3be800b8f0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 shutdown_connections 2024-04-25T13:12:20.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bf4108a50 0x7f3bf419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3bdc0a3cd0 0x7f3bdc0a6190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bf4102a90 0x7f3bf41992d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 --2- 172.21.15.72:0/1082178284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bf4103450 0x7f3bf4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:20.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 >> 172.21.15.72:0/1082178284 conn(0x7f3bf4079ba0 msgr2=0x7f3bf41019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:20.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 shutdown_connections 2024-04-25T13:12:20.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:20.804+0000 7f3bf929a700 1 -- 172.21.15.72:0/1082178284 wait complete. 2024-04-25T13:12:20.808 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-25T13:12:21.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:21.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:21.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:21 smithi193 ceph-mon[32743]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:21.492 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:21.492 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:21.492 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:21 smithi072 ceph-mon[29199]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:21.521 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-04-25T13:12:19.143104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":568,"num_read_kb":403,"num_write":339,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180918,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383712,"kb_used_data":13692,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558112,"statfs":{"total":1151940427776,"available":1151547506688,"internally_reserved":0,"allocated":14020608,"data_stored":7639119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":182,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.996224"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152818+0000","last_change":"2024-04-25T13:11:05.285490+0000","last_active":"2024-04-25T13:12:08.152818+0000","last_peered":"2024-04-25T13:12:08.152818+0000","last_clean":"2024-04-25T13:12:08.152818+0000","last_became_active":"2024-04-25T13:11:05.285066+0000","last_became_peered":"2024-04-25T13:11:05.285066+0000","last_unstale":"2024-04-25T13:12:08.152818+0000","last_undegraded":"2024-04-25T13:12:08.152818+0000","last_fullsized":"2024-04-25T13:12:08.152818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:30.002823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157208+0000","last_change":"2024-04-25T13:11:05.292670+0000","last_active":"2024-04-25T13:12:08.157208+0000","last_peered":"2024-04-25T13:12:08.157208+0000","last_clean":"2024-04-25T13:12:08.157208+0000","last_became_active":"2024-04-25T13:11:05.292279+0000","last_became_peered":"2024-04-25T13:11:05.292279+0000","last_unstale":"2024-04-25T13:12:08.157208+0000","last_undegraded":"2024-04-25T13:12:08.157208+0000","last_fullsized":"2024-04-25T13:12:08.157208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:26.439874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156949+0000","last_change":"2024-04-25T13:11:05.284228+0000","last_active":"2024-04-25T13:12:08.156949+0000","last_peered":"2024-04-25T13:12:08.156949+0000","last_clean":"2024-04-25T13:12:08.156949+0000","last_became_active":"2024-04-25T13:11:05.283877+0000","last_became_peered":"2024-04-25T13:11:05.283877+0000","last_unstale":"2024-04-25T13:12:08.156949+0000","last_undegraded":"2024-04-25T13:12:08.156949+0000","last_fullsized":"2024-04-25T13:12:08.156949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:19.768393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.333331+0000","last_change":"2024-04-25T13:11:05.276516+0000","last_active":"2024-04-25T13:12:08.333331+0000","last_peered":"2024-04-25T13:12:08.333331+0000","last_clean":"2024-04-25T13:12:08.333331+0000","last_became_active":"2024-04-25T13:11:05.276174+0000","last_became_peered":"2024-04-25T13:11:05.276174+0000","last_unstale":"2024-04-25T13:12:08.333331+0000","last_undegraded":"2024-04-25T13:12:08.333331+0000","last_fullsized":"2024-04-25T13:12:08.333331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:27.801716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152165+0000","last_change":"2024-04-25T13:11:05.279010+0000","last_active":"2024-04-25T13:12:08.152165+0000","last_peered":"2024-04-25T13:12:08.152165+0000","last_clean":"2024-04-25T13:12:08.152165+0000","last_became_active":"2024-04-25T13:11:05.276879+0000","last_became_peered":"2024-04-25T13:11:05.276879+0000","last_unstale":"2024-04-25T13:12:08.152165+0000","last_undegraded":"2024-04-25T13:12:08.152165+0000","last_fullsized":"2024-04-25T13:12:08.152165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:14.191090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151026+0000","last_change":"2024-04-25T13:11:05.291530+0000","last_active":"2024-04-25T13:12:08.151026+0000","last_peered":"2024-04-25T13:12:08.151026+0000","last_clean":"2024-04-25T13:12:08.151026+0000","last_became_active":"2024-04-25T13:11:05.291119+0000","last_became_peered":"2024-04-25T13:11:05.291119+0000","last_unstale":"2024-04-25T13:12:08.151026+0000","last_undegraded":"2024-04-25T13:12:08.151026+0000","last_fullsized":"2024-04-25T13:12:08.151026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:53.227468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148814+0000","last_change":"2024-04-25T13:11:05.281816+0000","last_active":"2024-04-25T13:12:08.148814+0000","last_peered":"2024-04-25T13:12:08.148814+0000","last_clean":"2024-04-25T13:12:08.148814+0000","last_became_active":"2024-04-25T13:11:05.281504+0000","last_became_peered":"2024-04-25T13:11:05.281504+0000","last_unstale":"2024-04-25T13:12:08.148814+0000","last_undegraded":"2024-04-25T13:12:08.148814+0000","last_fullsized":"2024-04-25T13:12:08.148814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.724238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157171+0000","last_change":"2024-04-25T13:11:05.279320+0000","last_active":"2024-04-25T13:12:08.157171+0000","last_peered":"2024-04-25T13:12:08.157171+0000","last_clean":"2024-04-25T13:12:08.157171+0000","last_became_active":"2024-04-25T13:11:05.278995+0000","last_became_peered":"2024-04-25T13:11:05.278995+0000","last_unstale":"2024-04-25T13:12:08.157171+0000","last_undegraded":"2024-04-25T13:12:08.157171+0000","last_fullsized":"2024-04-25T13:12:08.157171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:53.592529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.332057+0000","last_change":"2024-04-25T13:11:05.279109+0000","last_active":"2024-04-25T13:12:08.332057+0000","last_peered":"2024-04-25T13:12:08.332057+0000","last_clean":"2024-04-25T13:12:08.332057+0000","last_became_active":"2024-04-25T13:11:05.278523+0000","last_became_peered":"2024-04-25T13:11:05.278523+0000","last_unstale":"2024-04-25T13:12:08.332057+0000","last_undegraded":"2024-04-25T13:12:08.332057+0000","last_fullsized":"2024-04-25T13:12:08.332057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:24.646025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150843+0000","last_change":"2024-04-25T13:11:05.292039+0000","last_active":"2024-04-25T13:12:08.150843+0000","last_peered":"2024-04-25T13:12:08.150843+0000","last_clean":"2024-04-25T13:12:08.150843+0000","last_became_active":"2024-04-25T13:11:05.291649+0000","last_became_peered":"2024-04-25T13:11:05.291649+0000","last_unstale":"2024-04-25T13:12:08.150843+0000","last_undegraded":"2024-04-25T13:12:08.150843+0000","last_fullsized":"2024-04-25T13:12:08.150843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:00.696047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149115+0000","last_change":"2024-04-25T13:11:05.287067+0000","last_active":"2024-04-25T13:12:08.149115+0000","last_peered":"2024-04-25T13:12:08.149115+0000","last_clean":"2024-04-25T13:12:08.149115+0000","last_became_active":"2024-04-25T13:11:05.286664+0000","last_became_peered":"2024-04-25T13:11:05.286664+0000","last_unstale":"2024-04-25T13:12:08.149115+0000","last_undegraded":"2024-04-25T13:12:08.149115+0000","last_fullsized":"2024-04-25T13:12:08.149115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:12.677498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.333154+0000","last_change":"2024-04-25T13:11:05.293192+0000","last_active":"2024-04-25T13:12:08.333154+0000","last_peered":"2024-04-25T13:12:08.333154+0000","last_clean":"2024-04-25T13:12:08.333154+0000","last_became_active":"2024-04-25T13:11:05.292663+0000","last_became_peered":"2024-04-25T13:11:05.292663+0000","last_unstale":"2024-04-25T13:12:08.333154+0000","last_undegraded":"2024-04-25T13:12:08.333154+0000","last_fullsized":"2024-04-25T13:12:08.333154+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:14.942484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148531+0000","last_change":"2024-04-25T13:11:05.278090+0000","last_active":"2024-04-25T13:12:08.148531+0000","last_peered":"2024-04-25T13:12:08.148531+0000","last_clean":"2024-04-25T13:12:08.148531+0000","last_became_active":"2024-04-25T13:11:05.277744+0000","last_became_peered":"2024-04-25T13:11:05.277744+0000","last_unstale":"2024-04-25T13:12:08.148531+0000","last_undegraded":"2024-04-25T13:12:08.148531+0000","last_fullsized":"2024-04-25T13:12:08.148531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:49.311241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.158420+0000","last_change":"2024-04-25T13:11:05.277356+0000","last_active":"2024-04-25T13:12:08.158420+0000","last_peered":"2024-04-25T13:12:08.158420+0000","last_clean":"2024-04-25T13:12:08.158420+0000","last_became_active":"2024-04-25T13:11:05.276992+0000","last_became_peered":"2024-04-25T13:11:05.276992+0000","last_unstale":"2024-04-25T13:12:08.158420+0000","last_undegraded":"2024-04-25T13:12:08.158420+0000","last_fullsized":"2024-04-25T13:12:08.158420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:05.929613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156452+0000","last_change":"2024-04-25T13:11:05.271950+0000","last_active":"2024-04-25T13:12:08.156452+0000","last_peered":"2024-04-25T13:12:08.156452+0000","last_clean":"2024-04-25T13:12:08.156452+0000","last_became_active":"2024-04-25T13:11:05.271660+0000","last_became_peered":"2024-04-25T13:11:05.271660+0000","last_unstale":"2024-04-25T13:12:08.156452+0000","last_undegraded":"2024-04-25T13:12:08.156452+0000","last_fullsized":"2024-04-25T13:12:08.156452+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:33.534854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331140+0000","last_change":"2024-04-25T13:11:05.283689+0000","last_active":"2024-04-25T13:12:08.331140+0000","last_peered":"2024-04-25T13:12:08.331140+0000","last_clean":"2024-04-25T13:12:08.331140+0000","last_became_active":"2024-04-25T13:11:05.283364+0000","last_became_peered":"2024-04-25T13:11:05.283364+0000","last_unstale":"2024-04-25T13:12:08.331140+0000","last_undegraded":"2024-04-25T13:12:08.331140+0000","last_fullsized":"2024-04-25T13:12:08.331140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:24.109350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.153105+0000","last_change":"2024-04-25T13:11:05.280865+0000","last_active":"2024-04-25T13:12:08.153105+0000","last_peered":"2024-04-25T13:12:08.153105+0000","last_clean":"2024-04-25T13:12:08.153105+0000","last_became_active":"2024-04-25T13:11:05.280436+0000","last_became_peered":"2024-04-25T13:11:05.280436+0000","last_unstale":"2024-04-25T13:12:08.153105+0000","last_undegraded":"2024-04-25T13:12:08.153105+0000","last_fullsized":"2024-04-25T13:12:08.153105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:58.960578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151243+0000","last_change":"2024-04-25T13:11:05.282844+0000","last_active":"2024-04-25T13:12:08.151243+0000","last_peered":"2024-04-25T13:12:08.151243+0000","last_clean":"2024-04-25T13:12:08.151243+0000","last_became_active":"2024-04-25T13:11:05.282502+0000","last_became_peered":"2024-04-25T13:11:05.282502+0000","last_unstale":"2024-04-25T13:12:08.151243+0000","last_undegraded":"2024-04-25T13:12:08.151243+0000","last_fullsized":"2024-04-25T13:12:08.151243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:58.014902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.153464+0000","last_change":"2024-04-25T13:11:05.276419+0000","last_active":"2024-04-25T13:12:08.153464+0000","last_peered":"2024-04-25T13:12:08.153464+0000","last_clean":"2024-04-25T13:12:08.153464+0000","last_became_active":"2024-04-25T13:11:05.275953+0000","last_became_peered":"2024-04-25T13:11:05.275953+0000","last_unstale":"2024-04-25T13:12:08.153464+0000","last_undegraded":"2024-04-25T13:12:08.153464+0000","last_fullsized":"2024-04-25T13:12:08.153464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:41.993632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151167+0000","last_change":"2024-04-25T13:11:04.278345+0000","last_active":"2024-04-25T13:12:08.151167+0000","last_peered":"2024-04-25T13:12:08.151167+0000","last_clean":"2024-04-25T13:12:08.151167+0000","last_became_active":"2024-04-25T13:11:04.278017+0000","last_became_peered":"2024-04-25T13:11:04.278017+0000","last_unstale":"2024-04-25T13:12:08.151167+0000","last_undegraded":"2024-04-25T13:12:08.151167+0000","last_fullsized":"2024-04-25T13:12:08.151167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:34.496703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.155597+0000","last_change":"2024-04-25T13:12:08.155113+0000","last_active":"2024-04-25T13:12:08.155597+0000","last_peered":"2024-04-25T13:12:08.155597+0000","last_clean":"2024-04-25T13:12:08.155597+0000","last_became_active":"2024-04-25T13:12:08.154770+0000","last_became_peered":"2024-04-25T13:12:08.154770+0000","last_unstale":"2024-04-25T13:12:08.155597+0000","last_undegraded":"2024-04-25T13:12:08.155597+0000","last_fullsized":"2024-04-25T13:12:08.155597+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:08.759419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157627+0000","last_change":"2024-04-25T13:11:04.276017+0000","last_active":"2024-04-25T13:12:08.157627+0000","last_peered":"2024-04-25T13:12:08.157627+0000","last_clean":"2024-04-25T13:12:08.157627+0000","last_became_active":"2024-04-25T13:11:04.275718+0000","last_became_peered":"2024-04-25T13:11:04.275718+0000","last_unstale":"2024-04-25T13:12:08.157627+0000","last_undegraded":"2024-04-25T13:12:08.157627+0000","last_fullsized":"2024-04-25T13:12:08.157627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:45.850856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157591+0000","last_change":"2024-04-25T13:11:05.283185+0000","last_active":"2024-04-25T13:12:08.157591+0000","last_peered":"2024-04-25T13:12:08.157591+0000","last_clean":"2024-04-25T13:12:08.157591+0000","last_became_active":"2024-04-25T13:11:05.282876+0000","last_became_peered":"2024-04-25T13:11:05.282876+0000","last_unstale":"2024-04-25T13:12:08.157591+0000","last_undegraded":"2024-04-25T13:12:08.157591+0000","last_fullsized":"2024-04-25T13:12:08.157591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:08.403124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.341817+0000","last_change":"2024-04-25T13:12:08.341817+0000","last_active":"2024-04-25T13:12:08.341817+0000","last_peered":"2024-04-25T13:12:08.341817+0000","last_clean":"2024-04-25T13:12:08.341817+0000","last_became_active":"2024-04-25T13:12:08.333200+0000","last_became_peered":"2024-04-25T13:12:08.333200+0000","last_unstale":"2024-04-25T13:12:08.341817+0000","last_undegraded":"2024-04-25T13:12:08.341817+0000","last_fullsized":"2024-04-25T13:12:08.341817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:36.966208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,6],"acting":[8,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331408+0000","last_change":"2024-04-25T13:11:05.292858+0000","last_active":"2024-04-25T13:12:08.331408+0000","last_peered":"2024-04-25T13:12:08.331408+0000","last_clean":"2024-04-25T13:12:08.331408+0000","last_became_active":"2024-04-25T13:11:05.292474+0000","last_became_peered":"2024-04-25T13:11:05.292474+0000","last_unstale":"2024-04-25T13:12:08.331408+0000","last_undegraded":"2024-04-25T13:12:08.331408+0000","last_fullsized":"2024-04-25T13:12:08.331408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:46.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151545+0000","last_change":"2024-04-25T13:11:04.281104+0000","last_active":"2024-04-25T13:12:08.151545+0000","last_peered":"2024-04-25T13:12:08.151545+0000","last_clean":"2024-04-25T13:12:08.151545+0000","last_became_active":"2024-04-25T13:11:04.280644+0000","last_became_peered":"2024-04-25T13:11:04.280644+0000","last_unstale":"2024-04-25T13:12:08.151545+0000","last_undegraded":"2024-04-25T13:12:08.151545+0000","last_fullsized":"2024-04-25T13:12:08.151545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:15.938218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150298+0000","last_change":"2024-04-25T13:11:05.287532+0000","last_active":"2024-04-25T13:12:08.150298+0000","last_peered":"2024-04-25T13:12:08.150298+0000","last_clean":"2024-04-25T13:12:08.150298+0000","last_became_active":"2024-04-25T13:11:05.287225+0000","last_became_peered":"2024-04-25T13:11:05.287225+0000","last_unstale":"2024-04-25T13:12:08.150298+0000","last_undegraded":"2024-04-25T13:12:08.150298+0000","last_fullsized":"2024-04-25T13:12:08.150298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:23.531317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":96,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:15.393599+0000","last_change":"2024-04-25T13:11:04.277567+0000","last_active":"2024-04-25T13:12:15.393599+0000","last_peered":"2024-04-25T13:12:15.393599+0000","last_clean":"2024-04-25T13:12:15.393599+0000","last_became_active":"2024-04-25T13:11:04.277098+0000","last_became_peered":"2024-04-25T13:11:04.277098+0000","last_unstale":"2024-04-25T13:12:15.393599+0000","last_undegraded":"2024-04-25T13:12:15.393599+0000","last_fullsized":"2024-04-25T13:12:15.393599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:23.592989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151508+0000","last_change":"2024-04-25T13:11:05.271586+0000","last_active":"2024-04-25T13:12:08.151508+0000","last_peered":"2024-04-25T13:12:08.151508+0000","last_clean":"2024-04-25T13:12:08.151508+0000","last_became_active":"2024-04-25T13:11:05.271233+0000","last_became_peered":"2024-04-25T13:11:05.271233+0000","last_unstale":"2024-04-25T13:12:08.151508+0000","last_undegraded":"2024-04-25T13:12:08.151508+0000","last_fullsized":"2024-04-25T13:12:08.151508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:53.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.336520+0000","last_change":"2024-04-25T13:12:08.336520+0000","last_active":"2024-04-25T13:12:08.336520+0000","last_peered":"2024-04-25T13:12:08.336520+0000","last_clean":"2024-04-25T13:12:08.336520+0000","last_became_active":"2024-04-25T13:12:08.332742+0000","last_became_peered":"2024-04-25T13:12:08.332742+0000","last_unstale":"2024-04-25T13:12:08.336520+0000","last_undegraded":"2024-04-25T13:12:08.336520+0000","last_fullsized":"2024-04-25T13:12:08.336520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:25.092263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149279+0000","last_change":"2024-04-25T13:11:05.281360+0000","last_active":"2024-04-25T13:12:08.149279+0000","last_peered":"2024-04-25T13:12:08.149279+0000","last_clean":"2024-04-25T13:12:08.149279+0000","last_became_active":"2024-04-25T13:11:05.281048+0000","last_became_peered":"2024-04-25T13:11:05.281048+0000","last_unstale":"2024-04-25T13:12:08.149279+0000","last_undegraded":"2024-04-25T13:12:08.149279+0000","last_fullsized":"2024-04-25T13:12:08.149279+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:27.285125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148324+0000","last_change":"2024-04-25T13:11:04.272020+0000","last_active":"2024-04-25T13:12:08.148324+0000","last_peered":"2024-04-25T13:12:08.148324+0000","last_clean":"2024-04-25T13:12:08.148324+0000","last_became_active":"2024-04-25T13:11:04.271472+0000","last_became_peered":"2024-04-25T13:11:04.271472+0000","last_unstale":"2024-04-25T13:12:08.148324+0000","last_undegraded":"2024-04-25T13:12:08.148324+0000","last_fullsized":"2024-04-25T13:12:08.148324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:32.784185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157061+0000","last_change":"2024-04-25T13:11:05.272604+0000","last_active":"2024-04-25T13:12:08.157061+0000","last_peered":"2024-04-25T13:12:08.157061+0000","last_clean":"2024-04-25T13:12:08.157061+0000","last_became_active":"2024-04-25T13:11:05.272246+0000","last_became_peered":"2024-04-25T13:11:05.272246+0000","last_unstale":"2024-04-25T13:12:08.157061+0000","last_undegraded":"2024-04-25T13:12:08.157061+0000","last_fullsized":"2024-04-25T13:12:08.157061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:04.945342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149331+0000","last_change":"2024-04-25T13:11:04.279441+0000","last_active":"2024-04-25T13:12:08.149331+0000","last_peered":"2024-04-25T13:12:08.149331+0000","last_clean":"2024-04-25T13:12:08.149331+0000","last_became_active":"2024-04-25T13:11:04.278940+0000","last_became_peered":"2024-04-25T13:11:04.278940+0000","last_unstale":"2024-04-25T13:12:08.149331+0000","last_undegraded":"2024-04-25T13:12:08.149331+0000","last_fullsized":"2024-04-25T13:12:08.149331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:20.507906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151392+0000","last_change":"2024-04-25T13:11:05.282585+0000","last_active":"2024-04-25T13:12:08.151392+0000","last_peered":"2024-04-25T13:12:08.151392+0000","last_clean":"2024-04-25T13:12:08.151392+0000","last_became_active":"2024-04-25T13:11:05.282248+0000","last_became_peered":"2024-04-25T13:11:05.282248+0000","last_unstale":"2024-04-25T13:12:08.151392+0000","last_undegraded":"2024-04-25T13:12:08.151392+0000","last_fullsized":"2024-04-25T13:12:08.151392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.671230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":90,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:15.392911+0000","last_change":"2024-04-25T13:11:04.277079+0000","last_active":"2024-04-25T13:12:15.392911+0000","last_peered":"2024-04-25T13:12:15.392911+0000","last_clean":"2024-04-25T13:12:15.392911+0000","last_became_active":"2024-04-25T13:11:04.276694+0000","last_became_peered":"2024-04-25T13:11:04.276694+0000","last_unstale":"2024-04-25T13:12:15.392911+0000","last_undegraded":"2024-04-25T13:12:15.392911+0000","last_fullsized":"2024-04-25T13:12:15.392911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:36.714106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331876+0000","last_change":"2024-04-25T13:11:05.292920+0000","last_active":"2024-04-25T13:12:08.331876+0000","last_peered":"2024-04-25T13:12:08.331876+0000","last_clean":"2024-04-25T13:12:08.331876+0000","last_became_active":"2024-04-25T13:11:05.292556+0000","last_became_peered":"2024-04-25T13:11:05.292556+0000","last_unstale":"2024-04-25T13:12:08.331876+0000","last_undegraded":"2024-04-25T13:12:08.331876+0000","last_fullsized":"2024-04-25T13:12:08.331876+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:28.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148984+0000","last_change":"2024-04-25T13:11:04.277566+0000","last_active":"2024-04-25T13:12:08.148984+0000","last_peered":"2024-04-25T13:12:08.148984+0000","last_clean":"2024-04-25T13:12:08.148984+0000","last_became_active":"2024-04-25T13:11:04.276909+0000","last_became_peered":"2024-04-25T13:11:04.276909+0000","last_unstale":"2024-04-25T13:12:08.148984+0000","last_undegraded":"2024-04-25T13:12:08.148984+0000","last_fullsized":"2024-04-25T13:12:08.148984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:43.907256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157498+0000","last_change":"2024-04-25T13:11:05.273851+0000","last_active":"2024-04-25T13:12:08.157498+0000","last_peered":"2024-04-25T13:12:08.157498+0000","last_clean":"2024-04-25T13:12:08.157498+0000","last_became_active":"2024-04-25T13:11:05.273503+0000","last_became_peered":"2024-04-25T13:11:05.273503+0000","last_unstale":"2024-04-25T13:12:08.157498+0000","last_undegraded":"2024-04-25T13:12:08.157498+0000","last_fullsized":"2024-04-25T13:12:08.157498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:38.532019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150427+0000","last_change":"2024-04-25T13:11:04.279446+0000","last_active":"2024-04-25T13:12:08.150427+0000","last_peered":"2024-04-25T13:12:08.150427+0000","last_clean":"2024-04-25T13:12:08.150427+0000","last_became_active":"2024-04-25T13:11:04.279083+0000","last_became_peered":"2024-04-25T13:11:04.279083+0000","last_unstale":"2024-04-25T13:12:08.150427+0000","last_undegraded":"2024-04-25T13:12:08.150427+0000","last_fullsized":"2024-04-25T13:12:08.150427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:28.719965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331547+0000","last_change":"2024-04-25T13:11:05.283795+0000","last_active":"2024-04-25T13:12:08.331547+0000","last_peered":"2024-04-25T13:12:08.331547+0000","last_clean":"2024-04-25T13:12:08.331547+0000","last_became_active":"2024-04-25T13:11:05.283356+0000","last_became_peered":"2024-04-25T13:11:05.283356+0000","last_unstale":"2024-04-25T13:12:08.331547+0000","last_undegraded":"2024-04-25T13:12:08.331547+0000","last_fullsized":"2024-04-25T13:12:08.331547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:57.095913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'28","reported_seq":191,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:16.394259+0000","last_change":"2024-04-25T13:11:04.277100+0000","last_active":"2024-04-25T13:12:16.394259+0000","last_peered":"2024-04-25T13:12:16.394259+0000","last_clean":"2024-04-25T13:12:16.394259+0000","last_became_active":"2024-04-25T13:11:04.276699+0000","last_became_peered":"2024-04-25T13:11:04.276699+0000","last_unstale":"2024-04-25T13:12:16.394259+0000","last_undegraded":"2024-04-25T13:12:16.394259+0000","last_fullsized":"2024-04-25T13:12:16.394259+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:54.362024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":144,"num_read_kb":131,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152521+0000","last_change":"2024-04-25T13:11:05.284175+0000","last_active":"2024-04-25T13:12:08.152521+0000","last_peered":"2024-04-25T13:12:08.152521+0000","last_clean":"2024-04-25T13:12:08.152521+0000","last_became_active":"2024-04-25T13:11:05.283830+0000","last_became_peered":"2024-04-25T13:11:05.283830+0000","last_unstale":"2024-04-25T13:12:08.152521+0000","last_undegraded":"2024-04-25T13:12:08.152521+0000","last_fullsized":"2024-04-25T13:12:08.152521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:18.462117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152981+0000","last_change":"2024-04-25T13:11:04.277374+0000","last_active":"2024-04-25T13:12:08.152981+0000","last_peered":"2024-04-25T13:12:08.152981+0000","last_clean":"2024-04-25T13:12:08.152981+0000","last_became_active":"2024-04-25T13:11:04.276995+0000","last_became_peered":"2024-04-25T13:11:04.276995+0000","last_unstale":"2024-04-25T13:12:08.152981+0000","last_undegraded":"2024-04-25T13:12:08.152981+0000","last_fullsized":"2024-04-25T13:12:08.152981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:04.856506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148459+0000","last_change":"2024-04-25T13:11:05.284192+0000","last_active":"2024-04-25T13:12:08.148459+0000","last_peered":"2024-04-25T13:12:08.148459+0000","last_clean":"2024-04-25T13:12:08.148459+0000","last_became_active":"2024-04-25T13:11:05.283877+0000","last_became_peered":"2024-04-25T13:11:05.283877+0000","last_unstale":"2024-04-25T13:12:08.148459+0000","last_undegraded":"2024-04-25T13:12:08.148459+0000","last_fullsized":"2024-04-25T13:12:08.148459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:07.297999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149281+0000","last_change":"2024-04-25T13:11:04.276776+0000","last_active":"2024-04-25T13:12:08.149281+0000","last_peered":"2024-04-25T13:12:08.149281+0000","last_clean":"2024-04-25T13:12:08.149281+0000","last_became_active":"2024-04-25T13:11:04.276302+0000","last_became_peered":"2024-04-25T13:11:04.276302+0000","last_unstale":"2024-04-25T13:12:08.149281+0000","last_undegraded":"2024-04-25T13:12:08.149281+0000","last_fullsized":"2024-04-25T13:12:08.149281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:49.560839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149452+0000","last_change":"2024-04-25T13:11:05.281019+0000","last_active":"2024-04-25T13:12:08.149452+0000","last_peered":"2024-04-25T13:12:08.149452+0000","last_clean":"2024-04-25T13:12:08.149452+0000","last_became_active":"2024-04-25T13:11:05.280566+0000","last_became_peered":"2024-04-25T13:11:05.280566+0000","last_unstale":"2024-04-25T13:12:08.149452+0000","last_undegraded":"2024-04-25T13:12:08.149452+0000","last_fullsized":"2024-04-25T13:12:08.149452+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:34.163825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:14.391612+0000","last_change":"2024-04-25T13:11:04.274968+0000","last_active":"2024-04-25T13:12:14.391612+0000","last_peered":"2024-04-25T13:12:14.391612+0000","last_clean":"2024-04-25T13:12:14.391612+0000","last_became_active":"2024-04-25T13:11:04.274617+0000","last_became_peered":"2024-04-25T13:11:04.274617+0000","last_unstale":"2024-04-25T13:12:14.391612+0000","last_undegraded":"2024-04-25T13:12:14.391612+0000","last_fullsized":"2024-04-25T13:12:14.391612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.502451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148720+0000","last_change":"2024-04-25T13:11:05.284103+0000","last_active":"2024-04-25T13:12:08.148720+0000","last_peered":"2024-04-25T13:12:08.148720+0000","last_clean":"2024-04-25T13:12:08.148720+0000","last_became_active":"2024-04-25T13:11:05.283689+0000","last_became_peered":"2024-04-25T13:11:05.283689+0000","last_unstale":"2024-04-25T13:12:08.148720+0000","last_undegraded":"2024-04-25T13:12:08.148720+0000","last_fullsized":"2024-04-25T13:12:08.148720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:09.967889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.173200+0000","last_change":"2024-04-25T13:12:08.172732+0000","last_active":"2024-04-25T13:12:08.173200+0000","last_peered":"2024-04-25T13:12:08.173200+0000","last_clean":"2024-04-25T13:12:08.173200+0000","last_became_active":"2024-04-25T13:12:08.159083+0000","last_became_peered":"2024-04-25T13:12:08.159083+0000","last_unstale":"2024-04-25T13:12:08.173200+0000","last_undegraded":"2024-04-25T13:12:08.173200+0000","last_fullsized":"2024-04-25T13:12:08.173200+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:15.844731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149467+0000","last_change":"2024-04-25T13:11:05.283350+0000","last_active":"2024-04-25T13:12:08.149467+0000","last_peered":"2024-04-25T13:12:08.149467+0000","last_clean":"2024-04-25T13:12:08.149467+0000","last_became_active":"2024-04-25T13:11:05.282447+0000","last_became_peered":"2024-04-25T13:11:05.282447+0000","last_unstale":"2024-04-25T13:12:08.149467+0000","last_undegraded":"2024-04-25T13:12:08.149467+0000","last_fullsized":"2024-04-25T13:12:08.149467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:19.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151294+0000","last_change":"2024-04-25T13:11:04.275563+0000","last_active":"2024-04-25T13:12:08.151294+0000","last_peered":"2024-04-25T13:12:08.151294+0000","last_clean":"2024-04-25T13:12:08.151294+0000","last_became_active":"2024-04-25T13:11:04.275151+0000","last_became_peered":"2024-04-25T13:11:04.275151+0000","last_unstale":"2024-04-25T13:12:08.151294+0000","last_undegraded":"2024-04-25T13:12:08.151294+0000","last_fullsized":"2024-04-25T13:12:08.151294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:27.629485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152555+0000","last_change":"2024-04-25T13:11:05.280723+0000","last_active":"2024-04-25T13:12:08.152555+0000","last_peered":"2024-04-25T13:12:08.152555+0000","last_clean":"2024-04-25T13:12:08.152555+0000","last_became_active":"2024-04-25T13:11:05.280271+0000","last_became_peered":"2024-04-25T13:11:05.280271+0000","last_unstale":"2024-04-25T13:12:08.152555+0000","last_undegraded":"2024-04-25T13:12:08.152555+0000","last_fullsized":"2024-04-25T13:12:08.152555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:18.099610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151330+0000","last_change":"2024-04-25T13:11:04.275774+0000","last_active":"2024-04-25T13:12:08.151330+0000","last_peered":"2024-04-25T13:12:08.151330+0000","last_clean":"2024-04-25T13:12:08.151330+0000","last_became_active":"2024-04-25T13:11:04.275459+0000","last_became_peered":"2024-04-25T13:11:04.275459+0000","last_unstale":"2024-04-25T13:12:08.151330+0000","last_undegraded":"2024-04-25T13:12:08.151330+0000","last_fullsized":"2024-04-25T13:12:08.151330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.465906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150576+0000","last_change":"2024-04-25T13:11:05.285237+0000","last_active":"2024-04-25T13:12:08.150576+0000","last_peered":"2024-04-25T13:12:08.150576+0000","last_clean":"2024-04-25T13:12:08.150576+0000","last_became_active":"2024-04-25T13:11:05.284875+0000","last_became_peered":"2024-04-25T13:11:05.284875+0000","last_unstale":"2024-04-25T13:12:08.150576+0000","last_undegraded":"2024-04-25T13:12:08.150576+0000","last_fullsized":"2024-04-25T13:12:08.150576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:55.508632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148978+0000","last_change":"2024-04-25T13:11:04.273216+0000","last_active":"2024-04-25T13:12:08.148978+0000","last_peered":"2024-04-25T13:12:08.148978+0000","last_clean":"2024-04-25T13:12:08.148978+0000","last_became_active":"2024-04-25T13:11:04.272850+0000","last_became_peered":"2024-04-25T13:11:04.272850+0000","last_unstale":"2024-04-25T13:12:08.148978+0000","last_undegraded":"2024-04-25T13:12:08.148978+0000","last_fullsized":"2024-04-25T13:12:08.148978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:53.566971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150705+0000","last_change":"2024-04-25T13:11:05.289951+0000","last_active":"2024-04-25T13:12:08.150705+0000","last_peered":"2024-04-25T13:12:08.150705+0000","last_clean":"2024-04-25T13:12:08.150705+0000","last_became_active":"2024-04-25T13:11:05.289575+0000","last_became_peered":"2024-04-25T13:11:05.289575+0000","last_unstale":"2024-04-25T13:12:08.150705+0000","last_undegraded":"2024-04-25T13:12:08.150705+0000","last_fullsized":"2024-04-25T13:12:08.150705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:24.404489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156878+0000","last_change":"2024-04-25T13:12:08.156878+0000","last_active":"2024-04-25T13:12:08.156878+0000","last_peered":"2024-04-25T13:12:08.156878+0000","last_clean":"2024-04-25T13:12:08.156878+0000","last_became_active":"2024-04-25T13:12:08.156417+0000","last_became_peered":"2024-04-25T13:12:08.156417+0000","last_unstale":"2024-04-25T13:12:08.156878+0000","last_undegraded":"2024-04-25T13:12:08.156878+0000","last_fullsized":"2024-04-25T13:12:08.156878+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:48.449835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:15.392330+0000","last_change":"2024-04-25T13:11:04.277793+0000","last_active":"2024-04-25T13:12:15.392330+0000","last_peered":"2024-04-25T13:12:15.392330+0000","last_clean":"2024-04-25T13:12:15.392330+0000","last_became_active":"2024-04-25T13:11:04.277254+0000","last_became_peered":"2024-04-25T13:11:04.277254+0000","last_unstale":"2024-04-25T13:12:15.392330+0000","last_undegraded":"2024-04-25T13:12:15.392330+0000","last_fullsized":"2024-04-25T13:12:15.392330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:19.481074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157836+0000","last_change":"2024-04-25T13:11:05.285358+0000","last_active":"2024-04-25T13:12:08.157836+0000","last_peered":"2024-04-25T13:12:08.157836+0000","last_clean":"2024-04-25T13:12:08.157836+0000","last_became_active":"2024-04-25T13:11:05.284641+0000","last_became_peered":"2024-04-25T13:11:05.284641+0000","last_unstale":"2024-04-25T13:12:08.157836+0000","last_undegraded":"2024-04-25T13:12:08.157836+0000","last_fullsized":"2024-04-25T13:12:08.157836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:33.245556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.330524+0000","last_change":"2024-04-25T13:11:04.272294+0000","last_active":"2024-04-25T13:12:08.330524+0000","last_peered":"2024-04-25T13:12:08.330524+0000","last_clean":"2024-04-25T13:12:08.330524+0000","last_became_active":"2024-04-25T13:11:04.271898+0000","last_became_peered":"2024-04-25T13:11:04.271898+0000","last_unstale":"2024-04-25T13:12:08.330524+0000","last_undegraded":"2024-04-25T13:12:08.330524+0000","last_fullsized":"2024-04-25T13:12:08.330524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:29.130773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148896+0000","last_change":"2024-04-25T13:09:54.520980+0000","last_active":"2024-04-25T13:12:08.148896+0000","last_peered":"2024-04-25T13:12:08.148896+0000","last_clean":"2024-04-25T13:12:08.148896+0000","last_became_active":"2024-04-25T13:09:54.520446+0000","last_became_peered":"2024-04-25T13:09:54.520446+0000","last_unstale":"2024-04-25T13:12:08.148896+0000","last_undegraded":"2024-04-25T13:12:08.148896+0000","last_fullsized":"2024-04-25T13:12:08.148896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:06:33.347730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:06:33.347730+0000","last_clean_scrub_stamp":"2024-04-25T13:06:33.347730+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:00.996641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157461+0000","last_change":"2024-04-25T13:12:08.157461+0000","last_active":"2024-04-25T13:12:08.157461+0000","last_peered":"2024-04-25T13:12:08.157461+0000","last_clean":"2024-04-25T13:12:08.157461+0000","last_became_active":"2024-04-25T13:12:08.157197+0000","last_became_peered":"2024-04-25T13:12:08.157197+0000","last_unstale":"2024-04-25T13:12:08.157461+0000","last_undegraded":"2024-04-25T13:12:08.157461+0000","last_fullsized":"2024-04-25T13:12:08.157461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:16.164059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149442+0000","last_change":"2024-04-25T13:11:04.279131+0000","last_active":"2024-04-25T13:12:08.149442+0000","last_peered":"2024-04-25T13:12:08.149442+0000","last_clean":"2024-04-25T13:12:08.149442+0000","last_became_active":"2024-04-25T13:11:04.278801+0000","last_became_peered":"2024-04-25T13:11:04.278801+0000","last_unstale":"2024-04-25T13:12:08.149442+0000","last_undegraded":"2024-04-25T13:12:08.149442+0000","last_fullsized":"2024-04-25T13:12:08.149442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:06.341497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152604+0000","last_change":"2024-04-25T13:11:04.275784+0000","last_active":"2024-04-25T13:12:08.152604+0000","last_peered":"2024-04-25T13:12:08.152604+0000","last_clean":"2024-04-25T13:12:08.152604+0000","last_became_active":"2024-04-25T13:11:04.275487+0000","last_became_peered":"2024-04-25T13:11:04.275487+0000","last_unstale":"2024-04-25T13:12:08.152604+0000","last_undegraded":"2024-04-25T13:12:08.152604+0000","last_fullsized":"2024-04-25T13:12:08.152604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.405056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148771+0000","last_change":"2024-04-25T13:11:05.275854+0000","last_active":"2024-04-25T13:12:08.148771+0000","last_peered":"2024-04-25T13:12:08.148771+0000","last_clean":"2024-04-25T13:12:08.148771+0000","last_became_active":"2024-04-25T13:11:05.275527+0000","last_became_peered":"2024-04-25T13:11:05.275527+0000","last_unstale":"2024-04-25T13:12:08.148771+0000","last_undegraded":"2024-04-25T13:12:08.148771+0000","last_fullsized":"2024-04-25T13:12:08.148771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:42.300458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151275+0000","last_change":"2024-04-25T13:11:04.279415+0000","last_active":"2024-04-25T13:12:08.151275+0000","last_peered":"2024-04-25T13:12:08.151275+0000","last_clean":"2024-04-25T13:12:08.151275+0000","last_became_active":"2024-04-25T13:11:04.279119+0000","last_became_peered":"2024-04-25T13:11:04.279119+0000","last_unstale":"2024-04-25T13:12:08.151275+0000","last_undegraded":"2024-04-25T13:12:08.151275+0000","last_fullsized":"2024-04-25T13:12:08.151275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:18.917509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157756+0000","last_change":"2024-04-25T13:11:05.276317+0000","last_active":"2024-04-25T13:12:08.157756+0000","last_peered":"2024-04-25T13:12:08.157756+0000","last_clean":"2024-04-25T13:12:08.157756+0000","last_became_active":"2024-04-25T13:11:05.275781+0000","last_became_peered":"2024-04-25T13:11:05.275781+0000","last_unstale":"2024-04-25T13:12:08.157756+0000","last_undegraded":"2024-04-25T13:12:08.157756+0000","last_fullsized":"2024-04-25T13:12:08.157756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:41.628388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.330554+0000","last_change":"2024-04-25T13:11:04.273617+0000","last_active":"2024-04-25T13:12:08.330554+0000","last_peered":"2024-04-25T13:12:08.330554+0000","last_clean":"2024-04-25T13:12:08.330554+0000","last_became_active":"2024-04-25T13:11:04.272749+0000","last_became_peered":"2024-04-25T13:11:04.272749+0000","last_unstale":"2024-04-25T13:12:08.330554+0000","last_undegraded":"2024-04-25T13:12:08.330554+0000","last_fullsized":"2024-04-25T13:12:08.330554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:14.484215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157315+0000","last_change":"2024-04-25T13:11:05.274423+0000","last_active":"2024-04-25T13:12:08.157315+0000","last_peered":"2024-04-25T13:12:08.157315+0000","last_clean":"2024-04-25T13:12:08.157315+0000","last_became_active":"2024-04-25T13:11:05.273180+0000","last_became_peered":"2024-04-25T13:11:05.273180+0000","last_unstale":"2024-04-25T13:12:08.157315+0000","last_undegraded":"2024-04-25T13:12:08.157315+0000","last_fullsized":"2024-04-25T13:12:08.157315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:51.082000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.330170+0000","last_change":"2024-04-25T13:11:04.275671+0000","last_active":"2024-04-25T13:12:08.330170+0000","last_peered":"2024-04-25T13:12:08.330170+0000","last_clean":"2024-04-25T13:12:08.330170+0000","last_became_active":"2024-04-25T13:11:04.275357+0000","last_became_peered":"2024-04-25T13:11:04.275357+0000","last_unstale":"2024-04-25T13:12:08.330170+0000","last_undegraded":"2024-04-25T13:12:08.330170+0000","last_fullsized":"2024-04-25T13:12:08.330170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:55.552043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151111+0000","last_change":"2024-04-25T13:11:05.282327+0000","last_active":"2024-04-25T13:12:08.151111+0000","last_peered":"2024-04-25T13:12:08.151111+0000","last_clean":"2024-04-25T13:12:08.151111+0000","last_became_active":"2024-04-25T13:11:05.281903+0000","last_became_peered":"2024-04-25T13:11:05.281903+0000","last_unstale":"2024-04-25T13:12:08.151111+0000","last_undegraded":"2024-04-25T13:12:08.151111+0000","last_fullsized":"2024-04-25T13:12:08.151111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:40.490726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157427+0000","last_change":"2024-04-25T13:11:04.273065+0000","last_active":"2024-04-25T13:12:08.157427+0000","last_peered":"2024-04-25T13:12:08.157427+0000","last_clean":"2024-04-25T13:12:08.157427+0000","last_became_active":"2024-04-25T13:11:04.272723+0000","last_became_peered":"2024-04-25T13:11:04.272723+0000","last_unstale":"2024-04-25T13:12:08.157427+0000","last_undegraded":"2024-04-25T13:12:08.157427+0000","last_fullsized":"2024-04-25T13:12:08.157427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:56.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.154323+0000","last_change":"2024-04-25T13:11:05.276845+0000","last_active":"2024-04-25T13:12:08.154323+0000","last_peered":"2024-04-25T13:12:08.154323+0000","last_clean":"2024-04-25T13:12:08.154323+0000","last_became_active":"2024-04-25T13:11:05.276466+0000","last_became_peered":"2024-04-25T13:11:05.276466+0000","last_unstale":"2024-04-25T13:12:08.154323+0000","last_undegraded":"2024-04-25T13:12:08.154323+0000","last_fullsized":"2024-04-25T13:12:08.154323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:55.934335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152959+0000","last_change":"2024-04-25T13:11:04.278970+0000","last_active":"2024-04-25T13:12:08.152959+0000","last_peered":"2024-04-25T13:12:08.152959+0000","last_clean":"2024-04-25T13:12:08.152959+0000","last_became_active":"2024-04-25T13:11:04.278594+0000","last_became_peered":"2024-04-25T13:11:04.278594+0000","last_unstale":"2024-04-25T13:12:08.152959+0000","last_undegraded":"2024-04-25T13:12:08.152959+0000","last_fullsized":"2024-04-25T13:12:08.152959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:31.099566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.158053+0000","last_change":"2024-04-25T13:11:05.281008+0000","last_active":"2024-04-25T13:12:08.158053+0000","last_peered":"2024-04-25T13:12:08.158053+0000","last_clean":"2024-04-25T13:12:08.158053+0000","last_became_active":"2024-04-25T13:11:05.280581+0000","last_became_peered":"2024-04-25T13:11:05.280581+0000","last_unstale":"2024-04-25T13:12:08.158053+0000","last_undegraded":"2024-04-25T13:12:08.158053+0000","last_fullsized":"2024-04-25T13:12:08.158053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:28.636608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.337652+0000","last_change":"2024-04-25T13:12:08.337652+0000","last_active":"2024-04-25T13:12:08.337652+0000","last_peered":"2024-04-25T13:12:08.337652+0000","last_clean":"2024-04-25T13:12:08.337652+0000","last_became_active":"2024-04-25T13:12:08.333199+0000","last_became_peered":"2024-04-25T13:12:08.333199+0000","last_unstale":"2024-04-25T13:12:08.337652+0000","last_undegraded":"2024-04-25T13:12:08.337652+0000","last_fullsized":"2024-04-25T13:12:08.337652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:36.208674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149097+0000","last_change":"2024-04-25T13:11:05.288121+0000","last_active":"2024-04-25T13:12:08.149097+0000","last_peered":"2024-04-25T13:12:08.149097+0000","last_clean":"2024-04-25T13:12:08.149097+0000","last_became_active":"2024-04-25T13:11:05.287816+0000","last_became_peered":"2024-04-25T13:11:05.287816+0000","last_unstale":"2024-04-25T13:12:08.149097+0000","last_undegraded":"2024-04-25T13:12:08.149097+0000","last_fullsized":"2024-04-25T13:12:08.149097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:02.855646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.153888+0000","last_change":"2024-04-25T13:11:04.279883+0000","last_active":"2024-04-25T13:12:08.153888+0000","last_peered":"2024-04-25T13:12:08.153888+0000","last_clean":"2024-04-25T13:12:08.153888+0000","last_became_active":"2024-04-25T13:11:04.279408+0000","last_became_peered":"2024-04-25T13:11:04.279408+0000","last_unstale":"2024-04-25T13:12:08.153888+0000","last_undegraded":"2024-04-25T13:12:08.153888+0000","last_fullsized":"2024-04-25T13:12:08.153888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:42.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149240+0000","last_change":"2024-04-25T13:11:05.284845+0000","last_active":"2024-04-25T13:12:08.149240+0000","last_peered":"2024-04-25T13:12:08.149240+0000","last_clean":"2024-04-25T13:12:08.149240+0000","last_became_active":"2024-04-25T13:11:05.284300+0000","last_became_peered":"2024-04-25T13:11:05.284300+0000","last_unstale":"2024-04-25T13:12:08.149240+0000","last_undegraded":"2024-04-25T13:12:08.149240+0000","last_fullsized":"2024-04-25T13:12:08.149240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:54.233049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.162296+0000","last_change":"2024-04-25T13:12:08.161812+0000","last_active":"2024-04-25T13:12:08.162296+0000","last_peered":"2024-04-25T13:12:08.162296+0000","last_clean":"2024-04-25T13:12:08.162296+0000","last_became_active":"2024-04-25T13:12:08.158481+0000","last_became_peered":"2024-04-25T13:12:08.158481+0000","last_unstale":"2024-04-25T13:12:08.162296+0000","last_undegraded":"2024-04-25T13:12:08.162296+0000","last_fullsized":"2024-04-25T13:12:08.162296+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:09.330842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148918+0000","last_change":"2024-04-25T13:11:05.283181+0000","last_active":"2024-04-25T13:12:08.148918+0000","last_peered":"2024-04-25T13:12:08.148918+0000","last_clean":"2024-04-25T13:12:08.148918+0000","last_became_active":"2024-04-25T13:11:05.282848+0000","last_became_peered":"2024-04-25T13:11:05.282848+0000","last_unstale":"2024-04-25T13:12:08.148918+0000","last_undegraded":"2024-04-25T13:12:08.148918+0000","last_fullsized":"2024-04-25T13:12:08.148918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:21.372069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148646+0000","last_change":"2024-04-25T13:11:04.270691+0000","last_active":"2024-04-25T13:12:08.148646+0000","last_peered":"2024-04-25T13:12:08.148646+0000","last_clean":"2024-04-25T13:12:08.148646+0000","last_became_active":"2024-04-25T13:11:04.270337+0000","last_became_peered":"2024-04-25T13:11:04.270337+0000","last_unstale":"2024-04-25T13:12:08.148646+0000","last_undegraded":"2024-04-25T13:12:08.148646+0000","last_fullsized":"2024-04-25T13:12:08.148646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:05.632472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149329+0000","last_change":"2024-04-25T13:11:05.284284+0000","last_active":"2024-04-25T13:12:08.149329+0000","last_peered":"2024-04-25T13:12:08.149329+0000","last_clean":"2024-04-25T13:12:08.149329+0000","last_became_active":"2024-04-25T13:11:05.283952+0000","last_became_peered":"2024-04-25T13:11:05.283952+0000","last_unstale":"2024-04-25T13:12:08.149329+0000","last_undegraded":"2024-04-25T13:12:08.149329+0000","last_fullsized":"2024-04-25T13:12:08.149329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:32.396183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.154462+0000","last_change":"2024-04-25T13:11:05.278064+0000","last_active":"2024-04-25T13:12:08.154462+0000","last_peered":"2024-04-25T13:12:08.154462+0000","last_clean":"2024-04-25T13:12:08.154462+0000","last_became_active":"2024-04-25T13:11:05.277680+0000","last_became_peered":"2024-04-25T13:11:05.277680+0000","last_unstale":"2024-04-25T13:12:08.154462+0000","last_undegraded":"2024-04-25T13:12:08.154462+0000","last_fullsized":"2024-04-25T13:12:08.154462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:32.048667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.334359+0000","last_change":"2024-04-25T13:12:08.334359+0000","last_active":"2024-04-25T13:12:08.334359+0000","last_peered":"2024-04-25T13:12:08.334359+0000","last_clean":"2024-04-25T13:12:08.334359+0000","last_became_active":"2024-04-25T13:12:08.334066+0000","last_became_peered":"2024-04-25T13:12:08.334066+0000","last_unstale":"2024-04-25T13:12:08.334359+0000","last_undegraded":"2024-04-25T13:12:08.334359+0000","last_fullsized":"2024-04-25T13:12:08.334359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:18.060280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152837+0000","last_change":"2024-04-25T13:11:05.278072+0000","last_active":"2024-04-25T13:12:08.152837+0000","last_peered":"2024-04-25T13:12:08.152837+0000","last_clean":"2024-04-25T13:12:08.152837+0000","last_became_active":"2024-04-25T13:11:05.277720+0000","last_became_peered":"2024-04-25T13:11:05.277720+0000","last_unstale":"2024-04-25T13:12:08.152837+0000","last_undegraded":"2024-04-25T13:12:08.152837+0000","last_fullsized":"2024-04-25T13:12:08.152837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:14.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156777+0000","last_change":"2024-04-25T13:11:05.272611+0000","last_active":"2024-04-25T13:12:08.156777+0000","last_peered":"2024-04-25T13:12:08.156777+0000","last_clean":"2024-04-25T13:12:08.156777+0000","last_became_active":"2024-04-25T13:11:05.272318+0000","last_became_peered":"2024-04-25T13:11:05.272318+0000","last_unstale":"2024-04-25T13:12:08.156777+0000","last_undegraded":"2024-04-25T13:12:08.156777+0000","last_fullsized":"2024-04-25T13:12:08.156777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:52.796827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148800+0000","last_change":"2024-04-25T13:11:05.281146+0000","last_active":"2024-04-25T13:12:08.148800+0000","last_peered":"2024-04-25T13:12:08.148800+0000","last_clean":"2024-04-25T13:12:08.148800+0000","last_became_active":"2024-04-25T13:11:05.280897+0000","last_became_peered":"2024-04-25T13:11:05.280897+0000","last_unstale":"2024-04-25T13:12:08.148800+0000","last_undegraded":"2024-04-25T13:12:08.148800+0000","last_fullsized":"2024-04-25T13:12:08.148800+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:27.348411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.155162+0000","last_change":"2024-04-25T13:11:05.275482+0000","last_active":"2024-04-25T13:12:08.155162+0000","last_peered":"2024-04-25T13:12:08.155162+0000","last_clean":"2024-04-25T13:12:08.155162+0000","last_became_active":"2024-04-25T13:11:05.275066+0000","last_became_peered":"2024-04-25T13:11:05.275066+0000","last_unstale":"2024-04-25T13:12:08.155162+0000","last_undegraded":"2024-04-25T13:12:08.155162+0000","last_fullsized":"2024-04-25T13:12:08.155162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:27.286105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.153237+0000","last_change":"2024-04-25T13:11:05.287931+0000","last_active":"2024-04-25T13:12:08.153237+0000","last_peered":"2024-04-25T13:12:08.153237+0000","last_clean":"2024-04-25T13:12:08.153237+0000","last_became_active":"2024-04-25T13:11:05.287540+0000","last_became_peered":"2024-04-25T13:11:05.287540+0000","last_unstale":"2024-04-25T13:12:08.153237+0000","last_undegraded":"2024-04-25T13:12:08.153237+0000","last_fullsized":"2024-04-25T13:12:08.153237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:16.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156995+0000","last_change":"2024-04-25T13:11:05.274197+0000","last_active":"2024-04-25T13:12:08.156995+0000","last_peered":"2024-04-25T13:12:08.156995+0000","last_clean":"2024-04-25T13:12:08.156995+0000","last_became_active":"2024-04-25T13:11:05.273236+0000","last_became_peered":"2024-04-25T13:11:05.273236+0000","last_unstale":"2024-04-25T13:12:08.156995+0000","last_undegraded":"2024-04-25T13:12:08.156995+0000","last_fullsized":"2024-04-25T13:12:08.156995+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:55.471002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149340+0000","last_change":"2024-04-25T13:11:05.282693+0000","last_active":"2024-04-25T13:12:08.149340+0000","last_peered":"2024-04-25T13:12:08.149340+0000","last_clean":"2024-04-25T13:12:08.149340+0000","last_became_active":"2024-04-25T13:11:05.280002+0000","last_became_peered":"2024-04-25T13:11:05.280002+0000","last_unstale":"2024-04-25T13:12:08.149340+0000","last_undegraded":"2024-04-25T13:12:08.149340+0000","last_fullsized":"2024-04-25T13:12:08.149340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:08.661270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157330+0000","last_change":"2024-04-25T13:11:05.274581+0000","last_active":"2024-04-25T13:12:08.157330+0000","last_peered":"2024-04-25T13:12:08.157330+0000","last_clean":"2024-04-25T13:12:08.157330+0000","last_became_active":"2024-04-25T13:11:05.273634+0000","last_became_peered":"2024-04-25T13:11:05.273634+0000","last_unstale":"2024-04-25T13:12:08.157330+0000","last_undegraded":"2024-04-25T13:12:08.157330+0000","last_fullsized":"2024-04-25T13:12:08.157330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:13.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331507+0000","last_change":"2024-04-25T13:11:05.284581+0000","last_active":"2024-04-25T13:12:08.331507+0000","last_peered":"2024-04-25T13:12:08.331507+0000","last_clean":"2024-04-25T13:12:08.331507+0000","last_became_active":"2024-04-25T13:11:05.283032+0000","last_became_peered":"2024-04-25T13:11:05.283032+0000","last_unstale":"2024-04-25T13:12:08.331507+0000","last_undegraded":"2024-04-25T13:12:08.331507+0000","last_fullsized":"2024-04-25T13:12:08.331507+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:12.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148443+0000","last_change":"2024-04-25T13:11:05.285081+0000","last_active":"2024-04-25T13:12:08.148443+0000","last_peered":"2024-04-25T13:12:08.148443+0000","last_clean":"2024-04-25T13:12:08.148443+0000","last_became_active":"2024-04-25T13:11:05.283717+0000","last_became_peered":"2024-04-25T13:11:05.283717+0000","last_unstale":"2024-04-25T13:12:08.148443+0000","last_undegraded":"2024-04-25T13:12:08.148443+0000","last_fullsized":"2024-04-25T13:12:08.148443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:46.381684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152339+0000","last_change":"2024-04-25T13:11:05.278362+0000","last_active":"2024-04-25T13:12:08.152339+0000","last_peered":"2024-04-25T13:12:08.152339+0000","last_clean":"2024-04-25T13:12:08.152339+0000","last_became_active":"2024-04-25T13:11:05.277985+0000","last_became_peered":"2024-04-25T13:11:05.277985+0000","last_unstale":"2024-04-25T13:12:08.152339+0000","last_undegraded":"2024-04-25T13:12:08.152339+0000","last_fullsized":"2024-04-25T13:12:08.152339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:45.142805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":432,"num_read_kb":284,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":182,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285596,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":473445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547234,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32824,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712328,"statfs":{"total":95995035648,"available":95961423872,"internally_reserved":0,"allocated":1683456,"data_stored":1125790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":473499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005099,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":473307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1065349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":472888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":533727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1011712,"data_stored":474159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":535468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":473199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":473325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1605632,"data_stored":1064963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:12:21.525 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:12:21.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:21.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:21.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:21 smithi164 ceph-mon[32418]: from='mgr.14152 172.21.15.72:0/3778931582' entity='mgr.x' 2024-04-25T13:12:21.762 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:22.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:22 smithi193 ceph-mon[32743]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:12:22.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:22 smithi193 ceph-mon[32743]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 15 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:12:22.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:22 smithi072 ceph-mon[29199]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:12:22.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:22 smithi072 ceph-mon[29199]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 15 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:12:22.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:22 smithi164 ceph-mon[32418]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:12:22.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:22 smithi164 ceph-mon[32418]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 15 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:12:23.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.382+0000 7f7811f75700 1 -- 172.21.15.72:0/189800073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 msgr2=0x7f780c06bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:23.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.382+0000 7f7811f75700 1 --2- 172.21.15.72:0/189800073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 0x7f780c06bb50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7800009990 tx=0x7f7800009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.382+0000 7f7811f75700 1 -- 172.21.15.72:0/189800073 shutdown_connections 2024-04-25T13:12:23.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.382+0000 7f7811f75700 1 --2- 172.21.15.72:0/189800073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f780c105470 0x7f780c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.382+0000 7f7811f75700 1 --2- 172.21.15.72:0/189800073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f780c107960 0x7f780c06b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.382+0000 7f7811f75700 1 --2- 172.21.15.72:0/189800073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 0x7f780c06bb50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.382+0000 7f7811f75700 1 -- 172.21.15.72:0/189800073 >> 172.21.15.72:0/189800073 conn(0x7f780c078910 msgr2=0x7f780c078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.382+0000 7f7811f75700 1 -- 172.21.15.72:0/189800073 shutdown_connections 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.383+0000 7f7811f75700 1 -- 172.21.15.72:0/189800073 wait complete. 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.383+0000 7f7811f75700 1 Processor -- start 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.383+0000 7f7811f75700 1 -- start start 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7811f75700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 0x7f780c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7811f75700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f780c105470 0x7f780c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7811f75700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f780c107960 0x7f780c1a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7811f75700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1191f0 con 0x7f780c06b6d0 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7811f75700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f780c119070 con 0x7f780c107960 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7811f75700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f780c119370 con 0x7f780c105470 2024-04-25T13:12:23.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 0x7f780c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 0x7f780c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43084/0 (socket says 172.21.15.72:43084) 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780b7fe700 1 -- 172.21.15.72:0/1194431491 learned_addr learned my addr 172.21.15.72:0/1194431491 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780affd700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f780c105470 0x7f780c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780bfff700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f780c107960 0x7f780c1a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780b7fe700 1 -- 172.21.15.72:0/1194431491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f780c105470 msgr2=0x7f780c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780b7fe700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f780c105470 0x7f780c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780b7fe700 1 -- 172.21.15.72:0/1194431491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f780c107960 msgr2=0x7f780c1a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780b7fe700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f780c107960 0x7f780c1a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780b7fe700 1 -- 172.21.15.72:0/1194431491 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c1a4710 con 0x7f780c06b6d0 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780bfff700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f780c107960 0x7f780c1a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f780b7fe700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 0x7f780c19f700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c920 tx=0x7f77fc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7808ff9700 1 -- 172.21.15.72:0/1194431491 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc007b00 con 0x7f780c06b6d0 2024-04-25T13:12:23.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780c1a4a00 con 0x7f780c06b6d0 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7808ff9700 1 -- 172.21.15.72:0/1194431491 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77fc004ce0 con 0x7f780c06b6d0 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7808ff9700 1 -- 172.21.15.72:0/1194431491 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc005500 con 0x7f780c06b6d0 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.384+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780c1ac280 con 0x7f780c06b6d0 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.385+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780c069730 con 0x7f780c06b6d0 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.386+0000 7f7808ff9700 1 -- 172.21.15.72:0/1194431491 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f77fc007ca0 con 0x7f780c06b6d0 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.387+0000 7f7808ff9700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f77f40a3cd0 0x7f77f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.387+0000 7f780affd700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f77f40a3cd0 0x7f77f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.387+0000 7f7808ff9700 1 -- 172.21.15.72:0/1194431491 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f77fc0d00b0 con 0x7f780c06b6d0 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.387+0000 7f780affd700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f77f40a3cd0 0x7f77f40a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f780000b5c0 tx=0x7f7800005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:23.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.392+0000 7f7808ff9700 1 -- 172.21.15.72:0/1194431491 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77fc014030 con 0x7f780c06b6d0 2024-04-25T13:12:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.588+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f780c104b60 con 0x7f77f40a3cd0 2024-04-25T13:12:23.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.595+0000 7f7808ff9700 1 -- 172.21.15.72:0/1194431491 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258724 (secure 0 0 0) 0x7f780c104b60 con 0x7f77f40a3cd0 2024-04-25T13:12:23.598 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:23.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.597+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f77f40a3cd0 msgr2=0x7f77f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:23.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f77f40a3cd0 0x7f77f40a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f780000b5c0 tx=0x7f7800005960 comp rx=0 tx=0).stop 2024-04-25T13:12:23.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 msgr2=0x7f780c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:23.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 0x7f780c19f700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c920 tx=0x7f77fc00cc30 comp rx=0 tx=0).stop 2024-04-25T13:12:23.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 shutdown_connections 2024-04-25T13:12:23.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f780c107960 0x7f780c1a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f77f40a3cd0 0x7f77f40a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c06b6d0 0x7f780c19f700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 --2- 172.21.15.72:0/1194431491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f780c105470 0x7f780c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:23.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 >> 172.21.15.72:0/1194431491 conn(0x7f780c078910 msgr2=0x7f780c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:23.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 shutdown_connections 2024-04-25T13:12:23.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:23.598+0000 7f7811f75700 1 -- 172.21.15.72:0/1194431491 wait complete. 2024-04-25T13:12:23.601 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-25T13:12:24.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:24 smithi072 ceph-mon[29199]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:12:24.325 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-04-25T13:12:23.144143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":598,"num_read_kb":418,"num_write":339,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180918,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383712,"kb_used_data":13692,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558112,"statfs":{"total":1151940427776,"available":1151547506688,"internally_reserved":0,"allocated":14020608,"data_stored":7639119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003210"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152818+0000","last_change":"2024-04-25T13:11:05.285490+0000","last_active":"2024-04-25T13:12:08.152818+0000","last_peered":"2024-04-25T13:12:08.152818+0000","last_clean":"2024-04-25T13:12:08.152818+0000","last_became_active":"2024-04-25T13:11:05.285066+0000","last_became_peered":"2024-04-25T13:11:05.285066+0000","last_unstale":"2024-04-25T13:12:08.152818+0000","last_undegraded":"2024-04-25T13:12:08.152818+0000","last_fullsized":"2024-04-25T13:12:08.152818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:30.002823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157208+0000","last_change":"2024-04-25T13:11:05.292670+0000","last_active":"2024-04-25T13:12:08.157208+0000","last_peered":"2024-04-25T13:12:08.157208+0000","last_clean":"2024-04-25T13:12:08.157208+0000","last_became_active":"2024-04-25T13:11:05.292279+0000","last_became_peered":"2024-04-25T13:11:05.292279+0000","last_unstale":"2024-04-25T13:12:08.157208+0000","last_undegraded":"2024-04-25T13:12:08.157208+0000","last_fullsized":"2024-04-25T13:12:08.157208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:26.439874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156949+0000","last_change":"2024-04-25T13:11:05.284228+0000","last_active":"2024-04-25T13:12:08.156949+0000","last_peered":"2024-04-25T13:12:08.156949+0000","last_clean":"2024-04-25T13:12:08.156949+0000","last_became_active":"2024-04-25T13:11:05.283877+0000","last_became_peered":"2024-04-25T13:11:05.283877+0000","last_unstale":"2024-04-25T13:12:08.156949+0000","last_undegraded":"2024-04-25T13:12:08.156949+0000","last_fullsized":"2024-04-25T13:12:08.156949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:19.768393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.333331+0000","last_change":"2024-04-25T13:11:05.276516+0000","last_active":"2024-04-25T13:12:08.333331+0000","last_peered":"2024-04-25T13:12:08.333331+0000","last_clean":"2024-04-25T13:12:08.333331+0000","last_became_active":"2024-04-25T13:11:05.276174+0000","last_became_peered":"2024-04-25T13:11:05.276174+0000","last_unstale":"2024-04-25T13:12:08.333331+0000","last_undegraded":"2024-04-25T13:12:08.333331+0000","last_fullsized":"2024-04-25T13:12:08.333331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:27.801716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152165+0000","last_change":"2024-04-25T13:11:05.279010+0000","last_active":"2024-04-25T13:12:08.152165+0000","last_peered":"2024-04-25T13:12:08.152165+0000","last_clean":"2024-04-25T13:12:08.152165+0000","last_became_active":"2024-04-25T13:11:05.276879+0000","last_became_peered":"2024-04-25T13:11:05.276879+0000","last_unstale":"2024-04-25T13:12:08.152165+0000","last_undegraded":"2024-04-25T13:12:08.152165+0000","last_fullsized":"2024-04-25T13:12:08.152165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:14.191090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151026+0000","last_change":"2024-04-25T13:11:05.291530+0000","last_active":"2024-04-25T13:12:08.151026+0000","last_peered":"2024-04-25T13:12:08.151026+0000","last_clean":"2024-04-25T13:12:08.151026+0000","last_became_active":"2024-04-25T13:11:05.291119+0000","last_became_peered":"2024-04-25T13:11:05.291119+0000","last_unstale":"2024-04-25T13:12:08.151026+0000","last_undegraded":"2024-04-25T13:12:08.151026+0000","last_fullsized":"2024-04-25T13:12:08.151026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:53.227468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148814+0000","last_change":"2024-04-25T13:11:05.281816+0000","last_active":"2024-04-25T13:12:08.148814+0000","last_peered":"2024-04-25T13:12:08.148814+0000","last_clean":"2024-04-25T13:12:08.148814+0000","last_became_active":"2024-04-25T13:11:05.281504+0000","last_became_peered":"2024-04-25T13:11:05.281504+0000","last_unstale":"2024-04-25T13:12:08.148814+0000","last_undegraded":"2024-04-25T13:12:08.148814+0000","last_fullsized":"2024-04-25T13:12:08.148814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.724238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157171+0000","last_change":"2024-04-25T13:11:05.279320+0000","last_active":"2024-04-25T13:12:08.157171+0000","last_peered":"2024-04-25T13:12:08.157171+0000","last_clean":"2024-04-25T13:12:08.157171+0000","last_became_active":"2024-04-25T13:11:05.278995+0000","last_became_peered":"2024-04-25T13:11:05.278995+0000","last_unstale":"2024-04-25T13:12:08.157171+0000","last_undegraded":"2024-04-25T13:12:08.157171+0000","last_fullsized":"2024-04-25T13:12:08.157171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:53.592529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.332057+0000","last_change":"2024-04-25T13:11:05.279109+0000","last_active":"2024-04-25T13:12:08.332057+0000","last_peered":"2024-04-25T13:12:08.332057+0000","last_clean":"2024-04-25T13:12:08.332057+0000","last_became_active":"2024-04-25T13:11:05.278523+0000","last_became_peered":"2024-04-25T13:11:05.278523+0000","last_unstale":"2024-04-25T13:12:08.332057+0000","last_undegraded":"2024-04-25T13:12:08.332057+0000","last_fullsized":"2024-04-25T13:12:08.332057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:24.646025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150843+0000","last_change":"2024-04-25T13:11:05.292039+0000","last_active":"2024-04-25T13:12:08.150843+0000","last_peered":"2024-04-25T13:12:08.150843+0000","last_clean":"2024-04-25T13:12:08.150843+0000","last_became_active":"2024-04-25T13:11:05.291649+0000","last_became_peered":"2024-04-25T13:11:05.291649+0000","last_unstale":"2024-04-25T13:12:08.150843+0000","last_undegraded":"2024-04-25T13:12:08.150843+0000","last_fullsized":"2024-04-25T13:12:08.150843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:00.696047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149115+0000","last_change":"2024-04-25T13:11:05.287067+0000","last_active":"2024-04-25T13:12:08.149115+0000","last_peered":"2024-04-25T13:12:08.149115+0000","last_clean":"2024-04-25T13:12:08.149115+0000","last_became_active":"2024-04-25T13:11:05.286664+0000","last_became_peered":"2024-04-25T13:11:05.286664+0000","last_unstale":"2024-04-25T13:12:08.149115+0000","last_undegraded":"2024-04-25T13:12:08.149115+0000","last_fullsized":"2024-04-25T13:12:08.149115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:12.677498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.333154+0000","last_change":"2024-04-25T13:11:05.293192+0000","last_active":"2024-04-25T13:12:08.333154+0000","last_peered":"2024-04-25T13:12:08.333154+0000","last_clean":"2024-04-25T13:12:08.333154+0000","last_became_active":"2024-04-25T13:11:05.292663+0000","last_became_peered":"2024-04-25T13:11:05.292663+0000","last_unstale":"2024-04-25T13:12:08.333154+0000","last_undegraded":"2024-04-25T13:12:08.333154+0000","last_fullsized":"2024-04-25T13:12:08.333154+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:14.942484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148531+0000","last_change":"2024-04-25T13:11:05.278090+0000","last_active":"2024-04-25T13:12:08.148531+0000","last_peered":"2024-04-25T13:12:08.148531+0000","last_clean":"2024-04-25T13:12:08.148531+0000","last_became_active":"2024-04-25T13:11:05.277744+0000","last_became_peered":"2024-04-25T13:11:05.277744+0000","last_unstale":"2024-04-25T13:12:08.148531+0000","last_undegraded":"2024-04-25T13:12:08.148531+0000","last_fullsized":"2024-04-25T13:12:08.148531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:49.311241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.158420+0000","last_change":"2024-04-25T13:11:05.277356+0000","last_active":"2024-04-25T13:12:08.158420+0000","last_peered":"2024-04-25T13:12:08.158420+0000","last_clean":"2024-04-25T13:12:08.158420+0000","last_became_active":"2024-04-25T13:11:05.276992+0000","last_became_peered":"2024-04-25T13:11:05.276992+0000","last_unstale":"2024-04-25T13:12:08.158420+0000","last_undegraded":"2024-04-25T13:12:08.158420+0000","last_fullsized":"2024-04-25T13:12:08.158420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:05.929613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156452+0000","last_change":"2024-04-25T13:11:05.271950+0000","last_active":"2024-04-25T13:12:08.156452+0000","last_peered":"2024-04-25T13:12:08.156452+0000","last_clean":"2024-04-25T13:12:08.156452+0000","last_became_active":"2024-04-25T13:11:05.271660+0000","last_became_peered":"2024-04-25T13:11:05.271660+0000","last_unstale":"2024-04-25T13:12:08.156452+0000","last_undegraded":"2024-04-25T13:12:08.156452+0000","last_fullsized":"2024-04-25T13:12:08.156452+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:33.534854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331140+0000","last_change":"2024-04-25T13:11:05.283689+0000","last_active":"2024-04-25T13:12:08.331140+0000","last_peered":"2024-04-25T13:12:08.331140+0000","last_clean":"2024-04-25T13:12:08.331140+0000","last_became_active":"2024-04-25T13:11:05.283364+0000","last_became_peered":"2024-04-25T13:11:05.283364+0000","last_unstale":"2024-04-25T13:12:08.331140+0000","last_undegraded":"2024-04-25T13:12:08.331140+0000","last_fullsized":"2024-04-25T13:12:08.331140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:24.109350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.153105+0000","last_change":"2024-04-25T13:11:05.280865+0000","last_active":"2024-04-25T13:12:08.153105+0000","last_peered":"2024-04-25T13:12:08.153105+0000","last_clean":"2024-04-25T13:12:08.153105+0000","last_became_active":"2024-04-25T13:11:05.280436+0000","last_became_peered":"2024-04-25T13:11:05.280436+0000","last_unstale":"2024-04-25T13:12:08.153105+0000","last_undegraded":"2024-04-25T13:12:08.153105+0000","last_fullsized":"2024-04-25T13:12:08.153105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:58.960578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151243+0000","last_change":"2024-04-25T13:11:05.282844+0000","last_active":"2024-04-25T13:12:08.151243+0000","last_peered":"2024-04-25T13:12:08.151243+0000","last_clean":"2024-04-25T13:12:08.151243+0000","last_became_active":"2024-04-25T13:11:05.282502+0000","last_became_peered":"2024-04-25T13:11:05.282502+0000","last_unstale":"2024-04-25T13:12:08.151243+0000","last_undegraded":"2024-04-25T13:12:08.151243+0000","last_fullsized":"2024-04-25T13:12:08.151243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:58.014902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.153464+0000","last_change":"2024-04-25T13:11:05.276419+0000","last_active":"2024-04-25T13:12:08.153464+0000","last_peered":"2024-04-25T13:12:08.153464+0000","last_clean":"2024-04-25T13:12:08.153464+0000","last_became_active":"2024-04-25T13:11:05.275953+0000","last_became_peered":"2024-04-25T13:11:05.275953+0000","last_unstale":"2024-04-25T13:12:08.153464+0000","last_undegraded":"2024-04-25T13:12:08.153464+0000","last_fullsized":"2024-04-25T13:12:08.153464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:41.993632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151167+0000","last_change":"2024-04-25T13:11:04.278345+0000","last_active":"2024-04-25T13:12:08.151167+0000","last_peered":"2024-04-25T13:12:08.151167+0000","last_clean":"2024-04-25T13:12:08.151167+0000","last_became_active":"2024-04-25T13:11:04.278017+0000","last_became_peered":"2024-04-25T13:11:04.278017+0000","last_unstale":"2024-04-25T13:12:08.151167+0000","last_undegraded":"2024-04-25T13:12:08.151167+0000","last_fullsized":"2024-04-25T13:12:08.151167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:34.496703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.155597+0000","last_change":"2024-04-25T13:12:08.155113+0000","last_active":"2024-04-25T13:12:08.155597+0000","last_peered":"2024-04-25T13:12:08.155597+0000","last_clean":"2024-04-25T13:12:08.155597+0000","last_became_active":"2024-04-25T13:12:08.154770+0000","last_became_peered":"2024-04-25T13:12:08.154770+0000","last_unstale":"2024-04-25T13:12:08.155597+0000","last_undegraded":"2024-04-25T13:12:08.155597+0000","last_fullsized":"2024-04-25T13:12:08.155597+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:08.759419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157627+0000","last_change":"2024-04-25T13:11:04.276017+0000","last_active":"2024-04-25T13:12:08.157627+0000","last_peered":"2024-04-25T13:12:08.157627+0000","last_clean":"2024-04-25T13:12:08.157627+0000","last_became_active":"2024-04-25T13:11:04.275718+0000","last_became_peered":"2024-04-25T13:11:04.275718+0000","last_unstale":"2024-04-25T13:12:08.157627+0000","last_undegraded":"2024-04-25T13:12:08.157627+0000","last_fullsized":"2024-04-25T13:12:08.157627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:45.850856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157591+0000","last_change":"2024-04-25T13:11:05.283185+0000","last_active":"2024-04-25T13:12:08.157591+0000","last_peered":"2024-04-25T13:12:08.157591+0000","last_clean":"2024-04-25T13:12:08.157591+0000","last_became_active":"2024-04-25T13:11:05.282876+0000","last_became_peered":"2024-04-25T13:11:05.282876+0000","last_unstale":"2024-04-25T13:12:08.157591+0000","last_undegraded":"2024-04-25T13:12:08.157591+0000","last_fullsized":"2024-04-25T13:12:08.157591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:08.403124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.341817+0000","last_change":"2024-04-25T13:12:08.341817+0000","last_active":"2024-04-25T13:12:08.341817+0000","last_peered":"2024-04-25T13:12:08.341817+0000","last_clean":"2024-04-25T13:12:08.341817+0000","last_became_active":"2024-04-25T13:12:08.333200+0000","last_became_peered":"2024-04-25T13:12:08.333200+0000","last_unstale":"2024-04-25T13:12:08.341817+0000","last_undegraded":"2024-04-25T13:12:08.341817+0000","last_fullsized":"2024-04-25T13:12:08.341817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:36.966208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,6],"acting":[8,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331408+0000","last_change":"2024-04-25T13:11:05.292858+0000","last_active":"2024-04-25T13:12:08.331408+0000","last_peered":"2024-04-25T13:12:08.331408+0000","last_clean":"2024-04-25T13:12:08.331408+0000","last_became_active":"2024-04-25T13:11:05.292474+0000","last_became_peered":"2024-04-25T13:11:05.292474+0000","last_unstale":"2024-04-25T13:12:08.331408+0000","last_undegraded":"2024-04-25T13:12:08.331408+0000","last_fullsized":"2024-04-25T13:12:08.331408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:46.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151545+0000","last_change":"2024-04-25T13:11:04.281104+0000","last_active":"2024-04-25T13:12:08.151545+0000","last_peered":"2024-04-25T13:12:08.151545+0000","last_clean":"2024-04-25T13:12:08.151545+0000","last_became_active":"2024-04-25T13:11:04.280644+0000","last_became_peered":"2024-04-25T13:11:04.280644+0000","last_unstale":"2024-04-25T13:12:08.151545+0000","last_undegraded":"2024-04-25T13:12:08.151545+0000","last_fullsized":"2024-04-25T13:12:08.151545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:15.938218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150298+0000","last_change":"2024-04-25T13:11:05.287532+0000","last_active":"2024-04-25T13:12:08.150298+0000","last_peered":"2024-04-25T13:12:08.150298+0000","last_clean":"2024-04-25T13:12:08.150298+0000","last_became_active":"2024-04-25T13:11:05.287225+0000","last_became_peered":"2024-04-25T13:11:05.287225+0000","last_unstale":"2024-04-25T13:12:08.150298+0000","last_undegraded":"2024-04-25T13:12:08.150298+0000","last_fullsized":"2024-04-25T13:12:08.150298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:23.531317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":101,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:20.399978+0000","last_change":"2024-04-25T13:11:04.277567+0000","last_active":"2024-04-25T13:12:20.399978+0000","last_peered":"2024-04-25T13:12:20.399978+0000","last_clean":"2024-04-25T13:12:20.399978+0000","last_became_active":"2024-04-25T13:11:04.277098+0000","last_became_peered":"2024-04-25T13:11:04.277098+0000","last_unstale":"2024-04-25T13:12:20.399978+0000","last_undegraded":"2024-04-25T13:12:20.399978+0000","last_fullsized":"2024-04-25T13:12:20.399978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:23.592989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151508+0000","last_change":"2024-04-25T13:11:05.271586+0000","last_active":"2024-04-25T13:12:08.151508+0000","last_peered":"2024-04-25T13:12:08.151508+0000","last_clean":"2024-04-25T13:12:08.151508+0000","last_became_active":"2024-04-25T13:11:05.271233+0000","last_became_peered":"2024-04-25T13:11:05.271233+0000","last_unstale":"2024-04-25T13:12:08.151508+0000","last_undegraded":"2024-04-25T13:12:08.151508+0000","last_fullsized":"2024-04-25T13:12:08.151508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:53.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.336520+0000","last_change":"2024-04-25T13:12:08.336520+0000","last_active":"2024-04-25T13:12:08.336520+0000","last_peered":"2024-04-25T13:12:08.336520+0000","last_clean":"2024-04-25T13:12:08.336520+0000","last_became_active":"2024-04-25T13:12:08.332742+0000","last_became_peered":"2024-04-25T13:12:08.332742+0000","last_unstale":"2024-04-25T13:12:08.336520+0000","last_undegraded":"2024-04-25T13:12:08.336520+0000","last_fullsized":"2024-04-25T13:12:08.336520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:25.092263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149279+0000","last_change":"2024-04-25T13:11:05.281360+0000","last_active":"2024-04-25T13:12:08.149279+0000","last_peered":"2024-04-25T13:12:08.149279+0000","last_clean":"2024-04-25T13:12:08.149279+0000","last_became_active":"2024-04-25T13:11:05.281048+0000","last_became_peered":"2024-04-25T13:11:05.281048+0000","last_unstale":"2024-04-25T13:12:08.149279+0000","last_undegraded":"2024-04-25T13:12:08.149279+0000","last_fullsized":"2024-04-25T13:12:08.149279+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:27.285125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148324+0000","last_change":"2024-04-25T13:11:04.272020+0000","last_active":"2024-04-25T13:12:08.148324+0000","last_peered":"2024-04-25T13:12:08.148324+0000","last_clean":"2024-04-25T13:12:08.148324+0000","last_became_active":"2024-04-25T13:11:04.271472+0000","last_became_peered":"2024-04-25T13:11:04.271472+0000","last_unstale":"2024-04-25T13:12:08.148324+0000","last_undegraded":"2024-04-25T13:12:08.148324+0000","last_fullsized":"2024-04-25T13:12:08.148324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:32.784185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157061+0000","last_change":"2024-04-25T13:11:05.272604+0000","last_active":"2024-04-25T13:12:08.157061+0000","last_peered":"2024-04-25T13:12:08.157061+0000","last_clean":"2024-04-25T13:12:08.157061+0000","last_became_active":"2024-04-25T13:11:05.272246+0000","last_became_peered":"2024-04-25T13:11:05.272246+0000","last_unstale":"2024-04-25T13:12:08.157061+0000","last_undegraded":"2024-04-25T13:12:08.157061+0000","last_fullsized":"2024-04-25T13:12:08.157061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:04.945342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149331+0000","last_change":"2024-04-25T13:11:04.279441+0000","last_active":"2024-04-25T13:12:08.149331+0000","last_peered":"2024-04-25T13:12:08.149331+0000","last_clean":"2024-04-25T13:12:08.149331+0000","last_became_active":"2024-04-25T13:11:04.278940+0000","last_became_peered":"2024-04-25T13:11:04.278940+0000","last_unstale":"2024-04-25T13:12:08.149331+0000","last_undegraded":"2024-04-25T13:12:08.149331+0000","last_fullsized":"2024-04-25T13:12:08.149331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:20.507906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151392+0000","last_change":"2024-04-25T13:11:05.282585+0000","last_active":"2024-04-25T13:12:08.151392+0000","last_peered":"2024-04-25T13:12:08.151392+0000","last_clean":"2024-04-25T13:12:08.151392+0000","last_became_active":"2024-04-25T13:11:05.282248+0000","last_became_peered":"2024-04-25T13:11:05.282248+0000","last_unstale":"2024-04-25T13:12:08.151392+0000","last_undegraded":"2024-04-25T13:12:08.151392+0000","last_fullsized":"2024-04-25T13:12:08.151392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.671230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:20.399289+0000","last_change":"2024-04-25T13:11:04.277079+0000","last_active":"2024-04-25T13:12:20.399289+0000","last_peered":"2024-04-25T13:12:20.399289+0000","last_clean":"2024-04-25T13:12:20.399289+0000","last_became_active":"2024-04-25T13:11:04.276694+0000","last_became_peered":"2024-04-25T13:11:04.276694+0000","last_unstale":"2024-04-25T13:12:20.399289+0000","last_undegraded":"2024-04-25T13:12:20.399289+0000","last_fullsized":"2024-04-25T13:12:20.399289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:36.714106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331876+0000","last_change":"2024-04-25T13:11:05.292920+0000","last_active":"2024-04-25T13:12:08.331876+0000","last_peered":"2024-04-25T13:12:08.331876+0000","last_clean":"2024-04-25T13:12:08.331876+0000","last_became_active":"2024-04-25T13:11:05.292556+0000","last_became_peered":"2024-04-25T13:11:05.292556+0000","last_unstale":"2024-04-25T13:12:08.331876+0000","last_undegraded":"2024-04-25T13:12:08.331876+0000","last_fullsized":"2024-04-25T13:12:08.331876+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:28.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148984+0000","last_change":"2024-04-25T13:11:04.277566+0000","last_active":"2024-04-25T13:12:08.148984+0000","last_peered":"2024-04-25T13:12:08.148984+0000","last_clean":"2024-04-25T13:12:08.148984+0000","last_became_active":"2024-04-25T13:11:04.276909+0000","last_became_peered":"2024-04-25T13:11:04.276909+0000","last_unstale":"2024-04-25T13:12:08.148984+0000","last_undegraded":"2024-04-25T13:12:08.148984+0000","last_fullsized":"2024-04-25T13:12:08.148984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:43.907256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157498+0000","last_change":"2024-04-25T13:11:05.273851+0000","last_active":"2024-04-25T13:12:08.157498+0000","last_peered":"2024-04-25T13:12:08.157498+0000","last_clean":"2024-04-25T13:12:08.157498+0000","last_became_active":"2024-04-25T13:11:05.273503+0000","last_became_peered":"2024-04-25T13:11:05.273503+0000","last_unstale":"2024-04-25T13:12:08.157498+0000","last_undegraded":"2024-04-25T13:12:08.157498+0000","last_fullsized":"2024-04-25T13:12:08.157498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:38.532019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150427+0000","last_change":"2024-04-25T13:11:04.279446+0000","last_active":"2024-04-25T13:12:08.150427+0000","last_peered":"2024-04-25T13:12:08.150427+0000","last_clean":"2024-04-25T13:12:08.150427+0000","last_became_active":"2024-04-25T13:11:04.279083+0000","last_became_peered":"2024-04-25T13:11:04.279083+0000","last_unstale":"2024-04-25T13:12:08.150427+0000","last_undegraded":"2024-04-25T13:12:08.150427+0000","last_fullsized":"2024-04-25T13:12:08.150427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:28.719965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331547+0000","last_change":"2024-04-25T13:11:05.283795+0000","last_active":"2024-04-25T13:12:08.331547+0000","last_peered":"2024-04-25T13:12:08.331547+0000","last_clean":"2024-04-25T13:12:08.331547+0000","last_became_active":"2024-04-25T13:11:05.283356+0000","last_became_peered":"2024-04-25T13:11:05.283356+0000","last_unstale":"2024-04-25T13:12:08.331547+0000","last_undegraded":"2024-04-25T13:12:08.331547+0000","last_fullsized":"2024-04-25T13:12:08.331547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:57.095913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'28","reported_seq":201,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:21.400543+0000","last_change":"2024-04-25T13:11:04.277100+0000","last_active":"2024-04-25T13:12:21.400543+0000","last_peered":"2024-04-25T13:12:21.400543+0000","last_clean":"2024-04-25T13:12:21.400543+0000","last_became_active":"2024-04-25T13:11:04.276699+0000","last_became_peered":"2024-04-25T13:11:04.276699+0000","last_unstale":"2024-04-25T13:12:21.400543+0000","last_undegraded":"2024-04-25T13:12:21.400543+0000","last_fullsized":"2024-04-25T13:12:21.400543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:54.362024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":154,"num_read_kb":136,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152521+0000","last_change":"2024-04-25T13:11:05.284175+0000","last_active":"2024-04-25T13:12:08.152521+0000","last_peered":"2024-04-25T13:12:08.152521+0000","last_clean":"2024-04-25T13:12:08.152521+0000","last_became_active":"2024-04-25T13:11:05.283830+0000","last_became_peered":"2024-04-25T13:11:05.283830+0000","last_unstale":"2024-04-25T13:12:08.152521+0000","last_undegraded":"2024-04-25T13:12:08.152521+0000","last_fullsized":"2024-04-25T13:12:08.152521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:18.462117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152981+0000","last_change":"2024-04-25T13:11:04.277374+0000","last_active":"2024-04-25T13:12:08.152981+0000","last_peered":"2024-04-25T13:12:08.152981+0000","last_clean":"2024-04-25T13:12:08.152981+0000","last_became_active":"2024-04-25T13:11:04.276995+0000","last_became_peered":"2024-04-25T13:11:04.276995+0000","last_unstale":"2024-04-25T13:12:08.152981+0000","last_undegraded":"2024-04-25T13:12:08.152981+0000","last_fullsized":"2024-04-25T13:12:08.152981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:04.856506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148459+0000","last_change":"2024-04-25T13:11:05.284192+0000","last_active":"2024-04-25T13:12:08.148459+0000","last_peered":"2024-04-25T13:12:08.148459+0000","last_clean":"2024-04-25T13:12:08.148459+0000","last_became_active":"2024-04-25T13:11:05.283877+0000","last_became_peered":"2024-04-25T13:11:05.283877+0000","last_unstale":"2024-04-25T13:12:08.148459+0000","last_undegraded":"2024-04-25T13:12:08.148459+0000","last_fullsized":"2024-04-25T13:12:08.148459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:07.297999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149281+0000","last_change":"2024-04-25T13:11:04.276776+0000","last_active":"2024-04-25T13:12:08.149281+0000","last_peered":"2024-04-25T13:12:08.149281+0000","last_clean":"2024-04-25T13:12:08.149281+0000","last_became_active":"2024-04-25T13:11:04.276302+0000","last_became_peered":"2024-04-25T13:11:04.276302+0000","last_unstale":"2024-04-25T13:12:08.149281+0000","last_undegraded":"2024-04-25T13:12:08.149281+0000","last_fullsized":"2024-04-25T13:12:08.149281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:49.560839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149452+0000","last_change":"2024-04-25T13:11:05.281019+0000","last_active":"2024-04-25T13:12:08.149452+0000","last_peered":"2024-04-25T13:12:08.149452+0000","last_clean":"2024-04-25T13:12:08.149452+0000","last_became_active":"2024-04-25T13:11:05.280566+0000","last_became_peered":"2024-04-25T13:11:05.280566+0000","last_unstale":"2024-04-25T13:12:08.149452+0000","last_undegraded":"2024-04-25T13:12:08.149452+0000","last_fullsized":"2024-04-25T13:12:08.149452+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:34.163825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":96,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:19.398474+0000","last_change":"2024-04-25T13:11:04.274968+0000","last_active":"2024-04-25T13:12:19.398474+0000","last_peered":"2024-04-25T13:12:19.398474+0000","last_clean":"2024-04-25T13:12:19.398474+0000","last_became_active":"2024-04-25T13:11:04.274617+0000","last_became_peered":"2024-04-25T13:11:04.274617+0000","last_unstale":"2024-04-25T13:12:19.398474+0000","last_undegraded":"2024-04-25T13:12:19.398474+0000","last_fullsized":"2024-04-25T13:12:19.398474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.502451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148720+0000","last_change":"2024-04-25T13:11:05.284103+0000","last_active":"2024-04-25T13:12:08.148720+0000","last_peered":"2024-04-25T13:12:08.148720+0000","last_clean":"2024-04-25T13:12:08.148720+0000","last_became_active":"2024-04-25T13:11:05.283689+0000","last_became_peered":"2024-04-25T13:11:05.283689+0000","last_unstale":"2024-04-25T13:12:08.148720+0000","last_undegraded":"2024-04-25T13:12:08.148720+0000","last_fullsized":"2024-04-25T13:12:08.148720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:09.967889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.173200+0000","last_change":"2024-04-25T13:12:08.172732+0000","last_active":"2024-04-25T13:12:08.173200+0000","last_peered":"2024-04-25T13:12:08.173200+0000","last_clean":"2024-04-25T13:12:08.173200+0000","last_became_active":"2024-04-25T13:12:08.159083+0000","last_became_peered":"2024-04-25T13:12:08.159083+0000","last_unstale":"2024-04-25T13:12:08.173200+0000","last_undegraded":"2024-04-25T13:12:08.173200+0000","last_fullsized":"2024-04-25T13:12:08.173200+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:15.844731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149467+0000","last_change":"2024-04-25T13:11:05.283350+0000","last_active":"2024-04-25T13:12:08.149467+0000","last_peered":"2024-04-25T13:12:08.149467+0000","last_clean":"2024-04-25T13:12:08.149467+0000","last_became_active":"2024-04-25T13:11:05.282447+0000","last_became_peered":"2024-04-25T13:11:05.282447+0000","last_unstale":"2024-04-25T13:12:08.149467+0000","last_undegraded":"2024-04-25T13:12:08.149467+0000","last_fullsized":"2024-04-25T13:12:08.149467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:19.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151294+0000","last_change":"2024-04-25T13:11:04.275563+0000","last_active":"2024-04-25T13:12:08.151294+0000","last_peered":"2024-04-25T13:12:08.151294+0000","last_clean":"2024-04-25T13:12:08.151294+0000","last_became_active":"2024-04-25T13:11:04.275151+0000","last_became_peered":"2024-04-25T13:11:04.275151+0000","last_unstale":"2024-04-25T13:12:08.151294+0000","last_undegraded":"2024-04-25T13:12:08.151294+0000","last_fullsized":"2024-04-25T13:12:08.151294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:27.629485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152555+0000","last_change":"2024-04-25T13:11:05.280723+0000","last_active":"2024-04-25T13:12:08.152555+0000","last_peered":"2024-04-25T13:12:08.152555+0000","last_clean":"2024-04-25T13:12:08.152555+0000","last_became_active":"2024-04-25T13:11:05.280271+0000","last_became_peered":"2024-04-25T13:11:05.280271+0000","last_unstale":"2024-04-25T13:12:08.152555+0000","last_undegraded":"2024-04-25T13:12:08.152555+0000","last_fullsized":"2024-04-25T13:12:08.152555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:18.099610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151330+0000","last_change":"2024-04-25T13:11:04.275774+0000","last_active":"2024-04-25T13:12:08.151330+0000","last_peered":"2024-04-25T13:12:08.151330+0000","last_clean":"2024-04-25T13:12:08.151330+0000","last_became_active":"2024-04-25T13:11:04.275459+0000","last_became_peered":"2024-04-25T13:11:04.275459+0000","last_unstale":"2024-04-25T13:12:08.151330+0000","last_undegraded":"2024-04-25T13:12:08.151330+0000","last_fullsized":"2024-04-25T13:12:08.151330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.465906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150576+0000","last_change":"2024-04-25T13:11:05.285237+0000","last_active":"2024-04-25T13:12:08.150576+0000","last_peered":"2024-04-25T13:12:08.150576+0000","last_clean":"2024-04-25T13:12:08.150576+0000","last_became_active":"2024-04-25T13:11:05.284875+0000","last_became_peered":"2024-04-25T13:11:05.284875+0000","last_unstale":"2024-04-25T13:12:08.150576+0000","last_undegraded":"2024-04-25T13:12:08.150576+0000","last_fullsized":"2024-04-25T13:12:08.150576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:55.508632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148978+0000","last_change":"2024-04-25T13:11:04.273216+0000","last_active":"2024-04-25T13:12:08.148978+0000","last_peered":"2024-04-25T13:12:08.148978+0000","last_clean":"2024-04-25T13:12:08.148978+0000","last_became_active":"2024-04-25T13:11:04.272850+0000","last_became_peered":"2024-04-25T13:11:04.272850+0000","last_unstale":"2024-04-25T13:12:08.148978+0000","last_undegraded":"2024-04-25T13:12:08.148978+0000","last_fullsized":"2024-04-25T13:12:08.148978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:53.566971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.150705+0000","last_change":"2024-04-25T13:11:05.289951+0000","last_active":"2024-04-25T13:12:08.150705+0000","last_peered":"2024-04-25T13:12:08.150705+0000","last_clean":"2024-04-25T13:12:08.150705+0000","last_became_active":"2024-04-25T13:11:05.289575+0000","last_became_peered":"2024-04-25T13:11:05.289575+0000","last_unstale":"2024-04-25T13:12:08.150705+0000","last_undegraded":"2024-04-25T13:12:08.150705+0000","last_fullsized":"2024-04-25T13:12:08.150705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:24.404489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156878+0000","last_change":"2024-04-25T13:12:08.156878+0000","last_active":"2024-04-25T13:12:08.156878+0000","last_peered":"2024-04-25T13:12:08.156878+0000","last_clean":"2024-04-25T13:12:08.156878+0000","last_became_active":"2024-04-25T13:12:08.156417+0000","last_became_peered":"2024-04-25T13:12:08.156417+0000","last_unstale":"2024-04-25T13:12:08.156878+0000","last_undegraded":"2024-04-25T13:12:08.156878+0000","last_fullsized":"2024-04-25T13:12:08.156878+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:48.449835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":96,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:20.399133+0000","last_change":"2024-04-25T13:11:04.277793+0000","last_active":"2024-04-25T13:12:20.399133+0000","last_peered":"2024-04-25T13:12:20.399133+0000","last_clean":"2024-04-25T13:12:20.399133+0000","last_became_active":"2024-04-25T13:11:04.277254+0000","last_became_peered":"2024-04-25T13:11:04.277254+0000","last_unstale":"2024-04-25T13:12:20.399133+0000","last_undegraded":"2024-04-25T13:12:20.399133+0000","last_fullsized":"2024-04-25T13:12:20.399133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:19.481074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157836+0000","last_change":"2024-04-25T13:11:05.285358+0000","last_active":"2024-04-25T13:12:08.157836+0000","last_peered":"2024-04-25T13:12:08.157836+0000","last_clean":"2024-04-25T13:12:08.157836+0000","last_became_active":"2024-04-25T13:11:05.284641+0000","last_became_peered":"2024-04-25T13:11:05.284641+0000","last_unstale":"2024-04-25T13:12:08.157836+0000","last_undegraded":"2024-04-25T13:12:08.157836+0000","last_fullsized":"2024-04-25T13:12:08.157836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:33.245556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.330524+0000","last_change":"2024-04-25T13:11:04.272294+0000","last_active":"2024-04-25T13:12:08.330524+0000","last_peered":"2024-04-25T13:12:08.330524+0000","last_clean":"2024-04-25T13:12:08.330524+0000","last_became_active":"2024-04-25T13:11:04.271898+0000","last_became_peered":"2024-04-25T13:11:04.271898+0000","last_unstale":"2024-04-25T13:12:08.330524+0000","last_undegraded":"2024-04-25T13:12:08.330524+0000","last_fullsized":"2024-04-25T13:12:08.330524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:29.130773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148896+0000","last_change":"2024-04-25T13:09:54.520980+0000","last_active":"2024-04-25T13:12:08.148896+0000","last_peered":"2024-04-25T13:12:08.148896+0000","last_clean":"2024-04-25T13:12:08.148896+0000","last_became_active":"2024-04-25T13:09:54.520446+0000","last_became_peered":"2024-04-25T13:09:54.520446+0000","last_unstale":"2024-04-25T13:12:08.148896+0000","last_undegraded":"2024-04-25T13:12:08.148896+0000","last_fullsized":"2024-04-25T13:12:08.148896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:06:33.347730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:06:33.347730+0000","last_clean_scrub_stamp":"2024-04-25T13:06:33.347730+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:00.996641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157461+0000","last_change":"2024-04-25T13:12:08.157461+0000","last_active":"2024-04-25T13:12:08.157461+0000","last_peered":"2024-04-25T13:12:08.157461+0000","last_clean":"2024-04-25T13:12:08.157461+0000","last_became_active":"2024-04-25T13:12:08.157197+0000","last_became_peered":"2024-04-25T13:12:08.157197+0000","last_unstale":"2024-04-25T13:12:08.157461+0000","last_undegraded":"2024-04-25T13:12:08.157461+0000","last_fullsized":"2024-04-25T13:12:08.157461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:16.164059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149442+0000","last_change":"2024-04-25T13:11:04.279131+0000","last_active":"2024-04-25T13:12:08.149442+0000","last_peered":"2024-04-25T13:12:08.149442+0000","last_clean":"2024-04-25T13:12:08.149442+0000","last_became_active":"2024-04-25T13:11:04.278801+0000","last_became_peered":"2024-04-25T13:11:04.278801+0000","last_unstale":"2024-04-25T13:12:08.149442+0000","last_undegraded":"2024-04-25T13:12:08.149442+0000","last_fullsized":"2024-04-25T13:12:08.149442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:06.341497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152604+0000","last_change":"2024-04-25T13:11:04.275784+0000","last_active":"2024-04-25T13:12:08.152604+0000","last_peered":"2024-04-25T13:12:08.152604+0000","last_clean":"2024-04-25T13:12:08.152604+0000","last_became_active":"2024-04-25T13:11:04.275487+0000","last_became_peered":"2024-04-25T13:11:04.275487+0000","last_unstale":"2024-04-25T13:12:08.152604+0000","last_undegraded":"2024-04-25T13:12:08.152604+0000","last_fullsized":"2024-04-25T13:12:08.152604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.405056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148771+0000","last_change":"2024-04-25T13:11:05.275854+0000","last_active":"2024-04-25T13:12:08.148771+0000","last_peered":"2024-04-25T13:12:08.148771+0000","last_clean":"2024-04-25T13:12:08.148771+0000","last_became_active":"2024-04-25T13:11:05.275527+0000","last_became_peered":"2024-04-25T13:11:05.275527+0000","last_unstale":"2024-04-25T13:12:08.148771+0000","last_undegraded":"2024-04-25T13:12:08.148771+0000","last_fullsized":"2024-04-25T13:12:08.148771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:42.300458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151275+0000","last_change":"2024-04-25T13:11:04.279415+0000","last_active":"2024-04-25T13:12:08.151275+0000","last_peered":"2024-04-25T13:12:08.151275+0000","last_clean":"2024-04-25T13:12:08.151275+0000","last_became_active":"2024-04-25T13:11:04.279119+0000","last_became_peered":"2024-04-25T13:11:04.279119+0000","last_unstale":"2024-04-25T13:12:08.151275+0000","last_undegraded":"2024-04-25T13:12:08.151275+0000","last_fullsized":"2024-04-25T13:12:08.151275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:18.917509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157756+0000","last_change":"2024-04-25T13:11:05.276317+0000","last_active":"2024-04-25T13:12:08.157756+0000","last_peered":"2024-04-25T13:12:08.157756+0000","last_clean":"2024-04-25T13:12:08.157756+0000","last_became_active":"2024-04-25T13:11:05.275781+0000","last_became_peered":"2024-04-25T13:11:05.275781+0000","last_unstale":"2024-04-25T13:12:08.157756+0000","last_undegraded":"2024-04-25T13:12:08.157756+0000","last_fullsized":"2024-04-25T13:12:08.157756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:41.628388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.330554+0000","last_change":"2024-04-25T13:11:04.273617+0000","last_active":"2024-04-25T13:12:08.330554+0000","last_peered":"2024-04-25T13:12:08.330554+0000","last_clean":"2024-04-25T13:12:08.330554+0000","last_became_active":"2024-04-25T13:11:04.272749+0000","last_became_peered":"2024-04-25T13:11:04.272749+0000","last_unstale":"2024-04-25T13:12:08.330554+0000","last_undegraded":"2024-04-25T13:12:08.330554+0000","last_fullsized":"2024-04-25T13:12:08.330554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:14.484215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157315+0000","last_change":"2024-04-25T13:11:05.274423+0000","last_active":"2024-04-25T13:12:08.157315+0000","last_peered":"2024-04-25T13:12:08.157315+0000","last_clean":"2024-04-25T13:12:08.157315+0000","last_became_active":"2024-04-25T13:11:05.273180+0000","last_became_peered":"2024-04-25T13:11:05.273180+0000","last_unstale":"2024-04-25T13:12:08.157315+0000","last_undegraded":"2024-04-25T13:12:08.157315+0000","last_fullsized":"2024-04-25T13:12:08.157315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:51.082000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.330170+0000","last_change":"2024-04-25T13:11:04.275671+0000","last_active":"2024-04-25T13:12:08.330170+0000","last_peered":"2024-04-25T13:12:08.330170+0000","last_clean":"2024-04-25T13:12:08.330170+0000","last_became_active":"2024-04-25T13:11:04.275357+0000","last_became_peered":"2024-04-25T13:11:04.275357+0000","last_unstale":"2024-04-25T13:12:08.330170+0000","last_undegraded":"2024-04-25T13:12:08.330170+0000","last_fullsized":"2024-04-25T13:12:08.330170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:55.552043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.151111+0000","last_change":"2024-04-25T13:11:05.282327+0000","last_active":"2024-04-25T13:12:08.151111+0000","last_peered":"2024-04-25T13:12:08.151111+0000","last_clean":"2024-04-25T13:12:08.151111+0000","last_became_active":"2024-04-25T13:11:05.281903+0000","last_became_peered":"2024-04-25T13:11:05.281903+0000","last_unstale":"2024-04-25T13:12:08.151111+0000","last_undegraded":"2024-04-25T13:12:08.151111+0000","last_fullsized":"2024-04-25T13:12:08.151111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:40.490726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157427+0000","last_change":"2024-04-25T13:11:04.273065+0000","last_active":"2024-04-25T13:12:08.157427+0000","last_peered":"2024-04-25T13:12:08.157427+0000","last_clean":"2024-04-25T13:12:08.157427+0000","last_became_active":"2024-04-25T13:11:04.272723+0000","last_became_peered":"2024-04-25T13:11:04.272723+0000","last_unstale":"2024-04-25T13:12:08.157427+0000","last_undegraded":"2024-04-25T13:12:08.157427+0000","last_fullsized":"2024-04-25T13:12:08.157427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:56.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.154323+0000","last_change":"2024-04-25T13:11:05.276845+0000","last_active":"2024-04-25T13:12:08.154323+0000","last_peered":"2024-04-25T13:12:08.154323+0000","last_clean":"2024-04-25T13:12:08.154323+0000","last_became_active":"2024-04-25T13:11:05.276466+0000","last_became_peered":"2024-04-25T13:11:05.276466+0000","last_unstale":"2024-04-25T13:12:08.154323+0000","last_undegraded":"2024-04-25T13:12:08.154323+0000","last_fullsized":"2024-04-25T13:12:08.154323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:55.934335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152959+0000","last_change":"2024-04-25T13:11:04.278970+0000","last_active":"2024-04-25T13:12:08.152959+0000","last_peered":"2024-04-25T13:12:08.152959+0000","last_clean":"2024-04-25T13:12:08.152959+0000","last_became_active":"2024-04-25T13:11:04.278594+0000","last_became_peered":"2024-04-25T13:11:04.278594+0000","last_unstale":"2024-04-25T13:12:08.152959+0000","last_undegraded":"2024-04-25T13:12:08.152959+0000","last_fullsized":"2024-04-25T13:12:08.152959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:31.099566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.158053+0000","last_change":"2024-04-25T13:11:05.281008+0000","last_active":"2024-04-25T13:12:08.158053+0000","last_peered":"2024-04-25T13:12:08.158053+0000","last_clean":"2024-04-25T13:12:08.158053+0000","last_became_active":"2024-04-25T13:11:05.280581+0000","last_became_peered":"2024-04-25T13:11:05.280581+0000","last_unstale":"2024-04-25T13:12:08.158053+0000","last_undegraded":"2024-04-25T13:12:08.158053+0000","last_fullsized":"2024-04-25T13:12:08.158053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:28.636608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.337652+0000","last_change":"2024-04-25T13:12:08.337652+0000","last_active":"2024-04-25T13:12:08.337652+0000","last_peered":"2024-04-25T13:12:08.337652+0000","last_clean":"2024-04-25T13:12:08.337652+0000","last_became_active":"2024-04-25T13:12:08.333199+0000","last_became_peered":"2024-04-25T13:12:08.333199+0000","last_unstale":"2024-04-25T13:12:08.337652+0000","last_undegraded":"2024-04-25T13:12:08.337652+0000","last_fullsized":"2024-04-25T13:12:08.337652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:36.208674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149097+0000","last_change":"2024-04-25T13:11:05.288121+0000","last_active":"2024-04-25T13:12:08.149097+0000","last_peered":"2024-04-25T13:12:08.149097+0000","last_clean":"2024-04-25T13:12:08.149097+0000","last_became_active":"2024-04-25T13:11:05.287816+0000","last_became_peered":"2024-04-25T13:11:05.287816+0000","last_unstale":"2024-04-25T13:12:08.149097+0000","last_undegraded":"2024-04-25T13:12:08.149097+0000","last_fullsized":"2024-04-25T13:12:08.149097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:02.855646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.153888+0000","last_change":"2024-04-25T13:11:04.279883+0000","last_active":"2024-04-25T13:12:08.153888+0000","last_peered":"2024-04-25T13:12:08.153888+0000","last_clean":"2024-04-25T13:12:08.153888+0000","last_became_active":"2024-04-25T13:11:04.279408+0000","last_became_peered":"2024-04-25T13:11:04.279408+0000","last_unstale":"2024-04-25T13:12:08.153888+0000","last_undegraded":"2024-04-25T13:12:08.153888+0000","last_fullsized":"2024-04-25T13:12:08.153888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:42.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149240+0000","last_change":"2024-04-25T13:11:05.284845+0000","last_active":"2024-04-25T13:12:08.149240+0000","last_peered":"2024-04-25T13:12:08.149240+0000","last_clean":"2024-04-25T13:12:08.149240+0000","last_became_active":"2024-04-25T13:11:05.284300+0000","last_became_peered":"2024-04-25T13:11:05.284300+0000","last_unstale":"2024-04-25T13:12:08.149240+0000","last_undegraded":"2024-04-25T13:12:08.149240+0000","last_fullsized":"2024-04-25T13:12:08.149240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:54.233049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.162296+0000","last_change":"2024-04-25T13:12:08.161812+0000","last_active":"2024-04-25T13:12:08.162296+0000","last_peered":"2024-04-25T13:12:08.162296+0000","last_clean":"2024-04-25T13:12:08.162296+0000","last_became_active":"2024-04-25T13:12:08.158481+0000","last_became_peered":"2024-04-25T13:12:08.158481+0000","last_unstale":"2024-04-25T13:12:08.162296+0000","last_undegraded":"2024-04-25T13:12:08.162296+0000","last_fullsized":"2024-04-25T13:12:08.162296+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:09.330842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148918+0000","last_change":"2024-04-25T13:11:05.283181+0000","last_active":"2024-04-25T13:12:08.148918+0000","last_peered":"2024-04-25T13:12:08.148918+0000","last_clean":"2024-04-25T13:12:08.148918+0000","last_became_active":"2024-04-25T13:11:05.282848+0000","last_became_peered":"2024-04-25T13:11:05.282848+0000","last_unstale":"2024-04-25T13:12:08.148918+0000","last_undegraded":"2024-04-25T13:12:08.148918+0000","last_fullsized":"2024-04-25T13:12:08.148918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:21.372069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148646+0000","last_change":"2024-04-25T13:11:04.270691+0000","last_active":"2024-04-25T13:12:08.148646+0000","last_peered":"2024-04-25T13:12:08.148646+0000","last_clean":"2024-04-25T13:12:08.148646+0000","last_became_active":"2024-04-25T13:11:04.270337+0000","last_became_peered":"2024-04-25T13:11:04.270337+0000","last_unstale":"2024-04-25T13:12:08.148646+0000","last_undegraded":"2024-04-25T13:12:08.148646+0000","last_fullsized":"2024-04-25T13:12:08.148646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:03.251565+0000","last_clean_scrub_stamp":"2024-04-25T13:11:03.251565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:05.632472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149329+0000","last_change":"2024-04-25T13:11:05.284284+0000","last_active":"2024-04-25T13:12:08.149329+0000","last_peered":"2024-04-25T13:12:08.149329+0000","last_clean":"2024-04-25T13:12:08.149329+0000","last_became_active":"2024-04-25T13:11:05.283952+0000","last_became_peered":"2024-04-25T13:11:05.283952+0000","last_unstale":"2024-04-25T13:12:08.149329+0000","last_undegraded":"2024-04-25T13:12:08.149329+0000","last_fullsized":"2024-04-25T13:12:08.149329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:32.396183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.154462+0000","last_change":"2024-04-25T13:11:05.278064+0000","last_active":"2024-04-25T13:12:08.154462+0000","last_peered":"2024-04-25T13:12:08.154462+0000","last_clean":"2024-04-25T13:12:08.154462+0000","last_became_active":"2024-04-25T13:11:05.277680+0000","last_became_peered":"2024-04-25T13:11:05.277680+0000","last_unstale":"2024-04-25T13:12:08.154462+0000","last_undegraded":"2024-04-25T13:12:08.154462+0000","last_fullsized":"2024-04-25T13:12:08.154462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:32.048667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.334359+0000","last_change":"2024-04-25T13:12:08.334359+0000","last_active":"2024-04-25T13:12:08.334359+0000","last_peered":"2024-04-25T13:12:08.334359+0000","last_clean":"2024-04-25T13:12:08.334359+0000","last_became_active":"2024-04-25T13:12:08.334066+0000","last_became_peered":"2024-04-25T13:12:08.334066+0000","last_unstale":"2024-04-25T13:12:08.334359+0000","last_undegraded":"2024-04-25T13:12:08.334359+0000","last_fullsized":"2024-04-25T13:12:08.334359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:18.060280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152837+0000","last_change":"2024-04-25T13:11:05.278072+0000","last_active":"2024-04-25T13:12:08.152837+0000","last_peered":"2024-04-25T13:12:08.152837+0000","last_clean":"2024-04-25T13:12:08.152837+0000","last_became_active":"2024-04-25T13:11:05.277720+0000","last_became_peered":"2024-04-25T13:11:05.277720+0000","last_unstale":"2024-04-25T13:12:08.152837+0000","last_undegraded":"2024-04-25T13:12:08.152837+0000","last_fullsized":"2024-04-25T13:12:08.152837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:14.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156777+0000","last_change":"2024-04-25T13:11:05.272611+0000","last_active":"2024-04-25T13:12:08.156777+0000","last_peered":"2024-04-25T13:12:08.156777+0000","last_clean":"2024-04-25T13:12:08.156777+0000","last_became_active":"2024-04-25T13:11:05.272318+0000","last_became_peered":"2024-04-25T13:11:05.272318+0000","last_unstale":"2024-04-25T13:12:08.156777+0000","last_undegraded":"2024-04-25T13:12:08.156777+0000","last_fullsized":"2024-04-25T13:12:08.156777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:52.796827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148800+0000","last_change":"2024-04-25T13:11:05.281146+0000","last_active":"2024-04-25T13:12:08.148800+0000","last_peered":"2024-04-25T13:12:08.148800+0000","last_clean":"2024-04-25T13:12:08.148800+0000","last_became_active":"2024-04-25T13:11:05.280897+0000","last_became_peered":"2024-04-25T13:11:05.280897+0000","last_unstale":"2024-04-25T13:12:08.148800+0000","last_undegraded":"2024-04-25T13:12:08.148800+0000","last_fullsized":"2024-04-25T13:12:08.148800+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:27.348411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.155162+0000","last_change":"2024-04-25T13:11:05.275482+0000","last_active":"2024-04-25T13:12:08.155162+0000","last_peered":"2024-04-25T13:12:08.155162+0000","last_clean":"2024-04-25T13:12:08.155162+0000","last_became_active":"2024-04-25T13:11:05.275066+0000","last_became_peered":"2024-04-25T13:11:05.275066+0000","last_unstale":"2024-04-25T13:12:08.155162+0000","last_undegraded":"2024-04-25T13:12:08.155162+0000","last_fullsized":"2024-04-25T13:12:08.155162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:27.286105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.153237+0000","last_change":"2024-04-25T13:11:05.287931+0000","last_active":"2024-04-25T13:12:08.153237+0000","last_peered":"2024-04-25T13:12:08.153237+0000","last_clean":"2024-04-25T13:12:08.153237+0000","last_became_active":"2024-04-25T13:11:05.287540+0000","last_became_peered":"2024-04-25T13:11:05.287540+0000","last_unstale":"2024-04-25T13:12:08.153237+0000","last_undegraded":"2024-04-25T13:12:08.153237+0000","last_fullsized":"2024-04-25T13:12:08.153237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:16.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.156995+0000","last_change":"2024-04-25T13:11:05.274197+0000","last_active":"2024-04-25T13:12:08.156995+0000","last_peered":"2024-04-25T13:12:08.156995+0000","last_clean":"2024-04-25T13:12:08.156995+0000","last_became_active":"2024-04-25T13:11:05.273236+0000","last_became_peered":"2024-04-25T13:11:05.273236+0000","last_unstale":"2024-04-25T13:12:08.156995+0000","last_undegraded":"2024-04-25T13:12:08.156995+0000","last_fullsized":"2024-04-25T13:12:08.156995+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:55.471002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.149340+0000","last_change":"2024-04-25T13:11:05.282693+0000","last_active":"2024-04-25T13:12:08.149340+0000","last_peered":"2024-04-25T13:12:08.149340+0000","last_clean":"2024-04-25T13:12:08.149340+0000","last_became_active":"2024-04-25T13:11:05.280002+0000","last_became_peered":"2024-04-25T13:11:05.280002+0000","last_unstale":"2024-04-25T13:12:08.149340+0000","last_undegraded":"2024-04-25T13:12:08.149340+0000","last_fullsized":"2024-04-25T13:12:08.149340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:08.661270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.157330+0000","last_change":"2024-04-25T13:11:05.274581+0000","last_active":"2024-04-25T13:12:08.157330+0000","last_peered":"2024-04-25T13:12:08.157330+0000","last_clean":"2024-04-25T13:12:08.157330+0000","last_became_active":"2024-04-25T13:11:05.273634+0000","last_became_peered":"2024-04-25T13:11:05.273634+0000","last_unstale":"2024-04-25T13:12:08.157330+0000","last_undegraded":"2024-04-25T13:12:08.157330+0000","last_fullsized":"2024-04-25T13:12:08.157330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:13.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.331507+0000","last_change":"2024-04-25T13:11:05.284581+0000","last_active":"2024-04-25T13:12:08.331507+0000","last_peered":"2024-04-25T13:12:08.331507+0000","last_clean":"2024-04-25T13:12:08.331507+0000","last_became_active":"2024-04-25T13:11:05.283032+0000","last_became_peered":"2024-04-25T13:11:05.283032+0000","last_unstale":"2024-04-25T13:12:08.331507+0000","last_undegraded":"2024-04-25T13:12:08.331507+0000","last_fullsized":"2024-04-25T13:12:08.331507+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:12.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.148443+0000","last_change":"2024-04-25T13:11:05.285081+0000","last_active":"2024-04-25T13:12:08.148443+0000","last_peered":"2024-04-25T13:12:08.148443+0000","last_clean":"2024-04-25T13:12:08.148443+0000","last_became_active":"2024-04-25T13:11:05.283717+0000","last_became_peered":"2024-04-25T13:11:05.283717+0000","last_unstale":"2024-04-25T13:12:08.148443+0000","last_undegraded":"2024-04-25T13:12:08.148443+0000","last_fullsized":"2024-04-25T13:12:08.148443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:46.381684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:12:08.152339+0000","last_change":"2024-04-25T13:11:05.278362+0000","last_active":"2024-04-25T13:12:08.152339+0000","last_peered":"2024-04-25T13:12:08.152339+0000","last_clean":"2024-04-25T13:12:08.152339+0000","last_became_active":"2024-04-25T13:11:05.277985+0000","last_became_peered":"2024-04-25T13:11:05.277985+0000","last_unstale":"2024-04-25T13:12:08.152339+0000","last_undegraded":"2024-04-25T13:12:08.152339+0000","last_fullsized":"2024-04-25T13:12:08.152339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:11:04.253573+0000","last_clean_scrub_stamp":"2024-04-25T13:11:04.253573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:45.142805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":462,"num_read_kb":299,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":182,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285597,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":473445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547235,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32824,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712328,"statfs":{"total":95995035648,"available":95961423872,"internally_reserved":0,"allocated":1683456,"data_stored":1125790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":473499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005100,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":473307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1065349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":472888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":533727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1011712,"data_stored":474159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":535468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":473199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":473325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1605632,"data_stored":1064963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:12:24.329 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T13:12:24.329 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T13:12:24.329 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T13:12:24.329 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T13:12:24.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:24 smithi193 ceph-mon[32743]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:12:24.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:24 smithi164 ceph-mon[32418]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:12:24.569 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:25.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:25 smithi072 ceph-mon[29199]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:12:25.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:25 smithi193 ceph-mon[32743]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:12:25.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:25 smithi164 ceph-mon[32418]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:12:26.299 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:26 smithi072 ceph-mon[29199]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:26.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 -- 172.21.15.72:0/3762188197 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb0107940 msgr2=0x7febb0107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 --2- 172.21.15.72:0/3762188197 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb0107940 0x7febb0107d20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feba0009990 tx=0x7feba0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 -- 172.21.15.72:0/3762188197 shutdown_connections 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 --2- 172.21.15.72:0/3762188197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb0102390 0x7febb010e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 --2- 172.21.15.72:0/3762188197 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb0107940 0x7febb0107d20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 --2- 172.21.15.72:0/3762188197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb01019d0 0x7febb0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 -- 172.21.15.72:0/3762188197 >> 172.21.15.72:0/3762188197 conn(0x7febb0077990 msgr2=0x7febb0077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 -- 172.21.15.72:0/3762188197 shutdown_connections 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 -- 172.21.15.72:0/3762188197 wait complete. 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.290+0000 7febb69ff700 1 Processor -- start 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febb69ff700 1 -- start start 2024-04-25T13:12:26.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febb69ff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01019d0 0x7febb010f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:26.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febb69ff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb0102390 0x7febb010f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:26.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febb69ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb0107940 0x7febb010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:26.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febb69ff700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7febb0112ce0 con 0x7febb01019d0 2024-04-25T13:12:26.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febb69ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7febb0112b60 con 0x7febb0107940 2024-04-25T13:12:26.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febb69ff700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febb0112e60 con 0x7febb0102390 2024-04-25T13:12:26.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01019d0 0x7febb010f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:26.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaf7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb0102390 0x7febb010f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaf7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb0102390 0x7febb010f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.72:41312/0 (socket says 172.21.15.72:41312) 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01019d0 0x7febb010f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34418/0 (socket says 172.21.15.72:34418) 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaf7fe700 1 -- 172.21.15.72:0/3142410427 learned_addr learned my addr 172.21.15.72:0/3142410427 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febb4f9c700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb0107940 0x7febb010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaffff700 1 -- 172.21.15.72:0/3142410427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb0102390 msgr2=0x7febb010f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaffff700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb0102390 0x7febb010f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaffff700 1 -- 172.21.15.72:0/3142410427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb0107940 msgr2=0x7febb010fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaffff700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb0107940 0x7febb010fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.291+0000 7febaffff700 1 -- 172.21.15.72:0/3142410427 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb01a77e0 con 0x7febb01019d0 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.292+0000 7febaffff700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01019d0 0x7febb010f120 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feba00059d0 tx=0x7feba00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.292+0000 7febad7fa700 1 -- 172.21.15.72:0/3142410427 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0027070 con 0x7febb01019d0 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.292+0000 7febad7fa700 1 -- 172.21.15.72:0/3142410427 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba0003fa0 con 0x7febb01019d0 2024-04-25T13:12:26.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.292+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb01a7a70 con 0x7febb01019d0 2024-04-25T13:12:26.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.292+0000 7febad7fa700 1 -- 172.21.15.72:0/3142410427 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0022d00 con 0x7febb01019d0 2024-04-25T13:12:26.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.292+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb01a7ec0 con 0x7febb01019d0 2024-04-25T13:12:26.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.293+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb0069730 con 0x7febb01019d0 2024-04-25T13:12:26.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.294+0000 7febad7fa700 1 -- 172.21.15.72:0/3142410427 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feba002a070 con 0x7febb01019d0 2024-04-25T13:12:26.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.294+0000 7febad7fa700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7feb980a3cd0 0x7feb980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:26.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.294+0000 7febaf7fe700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7feb980a3cd0 0x7feb980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:26.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.294+0000 7febad7fa700 1 -- 172.21.15.72:0/3142410427 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7feba00d1540 con 0x7febb01019d0 2024-04-25T13:12:26.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.295+0000 7febaf7fe700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7feb980a3cd0 0x7feb980a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7febb00780b0 tx=0x7feba4009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:26.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.299+0000 7febad7fa700 1 -- 172.21.15.72:0/3142410427 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feba005eff0 con 0x7febb01019d0 2024-04-25T13:12:26.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:26 smithi193 ceph-mon[32743]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:26.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.544+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7febb002df80 con 0x7febb01019d0 2024-04-25T13:12:26.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.544+0000 7febad7fa700 1 -- 172.21.15.72:0/3142410427 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7feba0013070 con 0x7febb01019d0 2024-04-25T13:12:26.544 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:26.544 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T13:12:26.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7feb980a3cd0 msgr2=0x7feb980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:26.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7feb980a3cd0 0x7feb980a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7febb00780b0 tx=0x7feba4009290 comp rx=0 tx=0).stop 2024-04-25T13:12:26.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01019d0 msgr2=0x7febb010f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:26.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01019d0 0x7febb010f120 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feba00059d0 tx=0x7feba00047b0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 shutdown_connections 2024-04-25T13:12:26.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb0107940 0x7febb010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7feb980a3cd0 0x7feb980a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01019d0 0x7febb010f120 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 --2- 172.21.15.72:0/3142410427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb0102390 0x7febb010f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:26.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 >> 172.21.15.72:0/3142410427 conn(0x7febb0077990 msgr2=0x7febb00ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:26.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 shutdown_connections 2024-04-25T13:12:26.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:26.546+0000 7febb69ff700 1 -- 172.21.15.72:0/3142410427 wait complete. 2024-04-25T13:12:26.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:26 smithi164 ceph-mon[32418]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:27.161 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T13:12:27.162 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:12:27.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.263+0000 7f58f6b9b640 1 -- 172.21.15.72:0/3072772786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 msgr2=0x7f58f01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.263+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/3072772786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 0x7f58f01012c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009900 tx=0x7f58e402a410 comp rx=0 tx=0).stop 2024-04-25T13:12:27.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.263+0000 7f58f6b9b640 1 -- 172.21.15.72:0/3072772786 shutdown_connections 2024-04-25T13:12:27.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.263+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/3072772786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f01029b0 0x7f58f01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.263+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/3072772786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58f0101ff0 0x7f58f0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.263+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/3072772786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 0x7f58f01012c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.263+0000 7f58f6b9b640 1 -- 172.21.15.72:0/3072772786 >> 172.21.15.72:0/3072772786 conn(0x7f58f00fc430 msgr2=0x7f58f00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.263+0000 7f58f6b9b640 1 -- 172.21.15.72:0/3072772786 shutdown_connections 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.263+0000 7f58f6b9b640 1 -- 172.21.15.72:0/3072772786 wait complete. 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f6b9b640 1 Processor -- start 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f6b9b640 1 -- start start 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f6b9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 0x7f58f0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f6b9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f0101ff0 0x7f58f0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f6b9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58f01029b0 0x7f58f019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f6b9b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58f010c9f0 con 0x7f58f0100ea0 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f6b9b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58f010c870 con 0x7f58f01029b0 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f6b9b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58f010cb70 con 0x7f58f0101ff0 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f4937640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 0x7f58f0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f4937640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 0x7f58f0198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34428/0 (socket says 172.21.15.72:34428) 2024-04-25T13:12:27.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f4937640 1 -- 172.21.15.72:0/2046159597 learned_addr learned my addr 172.21.15.72:0/2046159597 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58effff640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f0101ff0 0x7f58f0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f4937640 1 -- 172.21.15.72:0/2046159597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f0101ff0 msgr2=0x7f58f0198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f4937640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f0101ff0 0x7f58f0198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f4937640 1 -- 172.21.15.72:0/2046159597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58f01029b0 msgr2=0x7f58f019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f4937640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58f01029b0 0x7f58f019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f4937640 1 -- 172.21.15.72:0/2046159597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f019fe60 con 0x7f58f0100ea0 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58effff640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f0101ff0 0x7f58f0198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.264+0000 7f58f4937640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 0x7f58f0198240 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f58e4005940 tx=0x7f58e4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.265+0000 7f58edffb640 1 -- 172.21.15.72:0/2046159597 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e403f030 con 0x7f58f0100ea0 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.265+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f01a00f0 con 0x7f58f0100ea0 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.265+0000 7f58edffb640 1 -- 172.21.15.72:0/2046159597 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e4003cb0 con 0x7f58f0100ea0 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.265+0000 7f58edffb640 1 -- 172.21.15.72:0/2046159597 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4035070 con 0x7f58f0100ea0 2024-04-25T13:12:27.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.265+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01a05e0 con 0x7f58f0100ea0 2024-04-25T13:12:27.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.266+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f01a0840 con 0x7f58f0100ea0 2024-04-25T13:12:27.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.267+0000 7f58edffb640 1 -- 172.21.15.72:0/2046159597 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f58e4042030 con 0x7f58f0100ea0 2024-04-25T13:12:27.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.267+0000 7f58edffb640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58e00a3c40 0x7f58e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.267+0000 7f58effff640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58e00a3c40 0x7f58e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.267+0000 7f58edffb640 1 -- 172.21.15.72:0/2046159597 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f58e40f0fa0 con 0x7f58f0100ea0 2024-04-25T13:12:27.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.268+0000 7f58effff640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58e00a3c40 0x7f58e00a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58f0199750 tx=0x7f58dc005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:27.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.272+0000 7f58edffb640 1 -- 172.21.15.72:0/2046159597 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58e4003e60 con 0x7f58f0100ea0 2024-04-25T13:12:27.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.450+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58f01a0bd0 con 0x7f58f0100ea0 2024-04-25T13:12:27.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.451+0000 7f58edffb640 1 -- 172.21.15.72:0/2046159597 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22740 (secure 0 0 0) 0x7f58e40bd810 con 0x7f58f0100ea0 2024-04-25T13:12:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:27.452 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:12:08.140624+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:12:27.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.453+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58e00a3c40 msgr2=0x7f58e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.453+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58e00a3c40 0x7f58e00a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58f0199750 tx=0x7f58dc005e70 comp rx=0 tx=0).stop 2024-04-25T13:12:27.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 msgr2=0x7f58f0198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 0x7f58f0198240 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f58e4005940 tx=0x7f58e4004430 comp rx=0 tx=0).stop 2024-04-25T13:12:27.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 shutdown_connections 2024-04-25T13:12:27.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58e00a3c40 0x7f58e00a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58f01029b0 0x7f58f019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f0101ff0 0x7f58f0198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 --2- 172.21.15.72:0/2046159597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58f0100ea0 0x7f58f0198240 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 >> 172.21.15.72:0/2046159597 conn(0x7f58f00fc430 msgr2=0x7f58f006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:27.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 shutdown_connections 2024-04-25T13:12:27.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.454+0000 7f58f6b9b640 1 -- 172.21.15.72:0/2046159597 wait complete. 2024-04-25T13:12:27.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:27 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3142410427' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:12:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:06:33.150709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:11:02.352089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:11:03.565033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:12:27.465 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:12:27.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:27 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3142410427' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:12:27.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:27 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3142410427' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:12:27.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.566+0000 7f0747adc640 1 -- 172.21.15.72:0/2054435451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07401029a0 msgr2=0x7f07401092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.566+0000 7f0747adc640 1 --2- 172.21.15.72:0/2054435451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07401029a0 0x7f07401092c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0738009ca0 tx=0x7f073802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 -- 172.21.15.72:0/2054435451 shutdown_connections 2024-04-25T13:12:27.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 --2- 172.21.15.72:0/2054435451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07401029a0 0x7f07401092c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 --2- 172.21.15.72:0/2054435451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0740101fe0 0x7f0740102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 --2- 172.21.15.72:0/2054435451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0740100e90 0x7f07401012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 -- 172.21.15.72:0/2054435451 >> 172.21.15.72:0/2054435451 conn(0x7f07400fc400 msgr2=0x7f07400fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:27.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 -- 172.21.15.72:0/2054435451 shutdown_connections 2024-04-25T13:12:27.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 -- 172.21.15.72:0/2054435451 wait complete. 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 Processor -- start 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 -- start start 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0740100e90 0x7f0740069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0740101fe0 0x7f074019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07401029a0 0x7f074019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f074010c9f0 con 0x7f0740100e90 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f074010c870 con 0x7f07401029a0 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0747adc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f074010cb70 con 0x7f0740101fe0 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0745878640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0740100e90 0x7f0740069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0745878640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0740100e90 0x7f0740069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34446/0 (socket says 172.21.15.72:34446) 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.567+0000 7f0745878640 1 -- 172.21.15.72:0/455594288 learned_addr learned my addr 172.21.15.72:0/455594288 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0745077640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0740101fe0 0x7f074019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0746079640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07401029a0 0x7f074019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0745878640 1 -- 172.21.15.72:0/455594288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0740101fe0 msgr2=0x7f074019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0745878640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0740101fe0 0x7f074019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0745878640 1 -- 172.21.15.72:0/455594288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07401029a0 msgr2=0x7f074019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0745878640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07401029a0 0x7f074019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0745878640 1 -- 172.21.15.72:0/455594288 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074019fe50 con 0x7f0740100e90 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0745077640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0740101fe0 0x7f074019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0745878640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0740100e90 0x7f0740069c50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f073000d770 tx=0x7f073000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0736ffd640 1 -- 172.21.15.72:0/455594288 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730017070 con 0x7f0740100e90 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0736ffd640 1 -- 172.21.15.72:0/455594288 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07300054d0 con 0x7f0740100e90 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07401a00e0 con 0x7f0740100e90 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0736ffd640 1 -- 172.21.15.72:0/455594288 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730004c30 con 0x7f0740100e90 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.568+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07401a05e0 con 0x7f0740100e90 2024-04-25T13:12:27.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.569+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07401081c0 con 0x7f0740100e90 2024-04-25T13:12:27.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.570+0000 7f0736ffd640 1 -- 172.21.15.72:0/455594288 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0730004dd0 con 0x7f0740100e90 2024-04-25T13:12:27.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.571+0000 7f0736ffd640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f072c0a3c40 0x7f072c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.571+0000 7f0745077640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f072c0a3c40 0x7f072c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.571+0000 7f0736ffd640 1 -- 172.21.15.72:0/455594288 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f07300d0550 con 0x7f0740100e90 2024-04-25T13:12:27.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.571+0000 7f0745077640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f072c0a3c40 0x7f072c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f073c005d00 tx=0x7f073c005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:27.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.575+0000 7f0736ffd640 1 -- 172.21.15.72:0/455594288 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f073009cdc0 con 0x7f0740100e90 2024-04-25T13:12:27.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.750+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f074002d830 con 0x7f0740100e90 2024-04-25T13:12:27.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.750+0000 7f0736ffd640 1 -- 172.21.15.72:0/455594288 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f073009c6c0 con 0x7f0740100e90 2024-04-25T13:12:27.750 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-25T13:12:27.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f072c0a3c40 msgr2=0x7f072c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f072c0a3c40 0x7f072c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f073c005d00 tx=0x7f073c005c70 comp rx=0 tx=0).stop 2024-04-25T13:12:27.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0740100e90 msgr2=0x7f0740069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0740100e90 0x7f0740069c50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f073000d770 tx=0x7f073000dc30 comp rx=0 tx=0).stop 2024-04-25T13:12:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 shutdown_connections 2024-04-25T13:12:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f072c0a3c40 0x7f072c0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07401029a0 0x7f074019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0740101fe0 0x7f074019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 --2- 172.21.15.72:0/455594288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0740100e90 0x7f0740069c50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 >> 172.21.15.72:0/455594288 conn(0x7f07400fc400 msgr2=0x7f0740105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 shutdown_connections 2024-04-25T13:12:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.752+0000 7f0747adc640 1 -- 172.21.15.72:0/455594288 wait complete. 2024-04-25T13:12:27.764 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:12:27.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 -- 172.21.15.72:0/4173973589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20103c10 msgr2=0x7fad2010a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 --2- 172.21.15.72:0/4173973589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20103c10 0x7fad2010a4a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad18009cd0 tx=0x7fad1802a8e0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 -- 172.21.15.72:0/4173973589 shutdown_connections 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 --2- 172.21.15.72:0/4173973589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20103c10 0x7fad2010a4a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 --2- 172.21.15.72:0/4173973589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad20103250 0x7fad201036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 --2- 172.21.15.72:0/4173973589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20100280 0x7fad201006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 -- 172.21.15.72:0/4173973589 >> 172.21.15.72:0/4173973589 conn(0x7fad200fbbb0 msgr2=0x7fad200fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 -- 172.21.15.72:0/4173973589 shutdown_connections 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 -- 172.21.15.72:0/4173973589 wait complete. 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.863+0000 7fad282ef640 1 Processor -- start 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad282ef640 1 -- start start 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad282ef640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20100280 0x7fad20198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad282ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20103250 0x7fad20198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad282ef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad20103c10 0x7fad2019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad282ef640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fad2010db20 con 0x7fad20100280 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad282ef640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad2010d9a0 con 0x7fad20103250 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad282ef640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad2010dca0 con 0x7fad20103c10 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2608b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20100280 0x7fad20198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2608b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20100280 0x7fad20198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34458/0 (socket says 172.21.15.72:34458) 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2608b640 1 -- 172.21.15.72:0/391802862 learned_addr learned my addr 172.21.15.72:0/391802862 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2588a640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20103250 0x7fad20198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2688c640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad20103c10 0x7fad2019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2608b640 1 -- 172.21.15.72:0/391802862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad20103c10 msgr2=0x7fad2019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2608b640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad20103c10 0x7fad2019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2608b640 1 -- 172.21.15.72:0/391802862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20103250 msgr2=0x7fad20198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2608b640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20103250 0x7fad20198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2608b640 1 -- 172.21.15.72:0/391802862 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2019fd60 con 0x7fad20100280 2024-04-25T13:12:27.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2588a640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20103250 0x7fad20198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:27.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad2608b640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20100280 0x7fad20198140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d770 tx=0x7fad1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:27.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad177fe640 1 -- 172.21.15.72:0/391802862 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10017070 con 0x7fad20100280 2024-04-25T13:12:27.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2019fff0 con 0x7fad20100280 2024-04-25T13:12:27.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad177fe640 1 -- 172.21.15.72:0/391802862 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad10009c30 con 0x7fad20100280 2024-04-25T13:12:27.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad177fe640 1 -- 172.21.15.72:0/391802862 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10003cc0 con 0x7fad20100280 2024-04-25T13:12:27.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.864+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad201a04b0 con 0x7fad20100280 2024-04-25T13:12:27.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.865+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad20109380 con 0x7fad20100280 2024-04-25T13:12:27.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.866+0000 7fad177fe640 1 -- 172.21.15.72:0/391802862 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fad10003e60 con 0x7fad20100280 2024-04-25T13:12:27.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.867+0000 7fad177fe640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad0c0a3c40 0x7fad0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:27.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.867+0000 7fad2588a640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad0c0a3c40 0x7fad0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:27.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.867+0000 7fad177fe640 1 -- 172.21.15.72:0/391802862 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fad100d0000 con 0x7fad20100280 2024-04-25T13:12:27.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.867+0000 7fad2588a640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad0c0a3c40 0x7fad0c0a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fad20199650 tx=0x7fad1c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:27.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:27.872+0000 7fad177fe640 1 -- 172.21.15.72:0/391802862 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad1009c870 con 0x7fad20100280 2024-04-25T13:12:28.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.049+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fad20062bf0 con 0x7fad20100280 2024-04-25T13:12:28.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.049+0000 7fad177fe640 1 -- 172.21.15.72:0/391802862 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fad10009de0 con 0x7fad20100280 2024-04-25T13:12:28.049 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-25T13:12:28.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad0c0a3c40 msgr2=0x7fad0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad0c0a3c40 0x7fad0c0a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fad20199650 tx=0x7fad1c005d30 comp rx=0 tx=0).stop 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20100280 msgr2=0x7fad20198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20100280 0x7fad20198140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d770 tx=0x7fad1000dc30 comp rx=0 tx=0).stop 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 shutdown_connections 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad0c0a3c40 0x7fad0c0a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad20103c10 0x7fad2019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20103250 0x7fad20198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 --2- 172.21.15.72:0/391802862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20100280 0x7fad20198140 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 >> 172.21.15.72:0/391802862 conn(0x7fad200fbbb0 msgr2=0x7fad20106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 shutdown_connections 2024-04-25T13:12:28.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.051+0000 7fad282ef640 1 -- 172.21.15.72:0/391802862 wait complete. 2024-04-25T13:12:28.062 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:12:28.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.163+0000 7f52f6b4f640 1 -- 172.21.15.72:0/2715922427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0102990 msgr2=0x7f52f01092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.163+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/2715922427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0102990 0x7f52f01092b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009ca0 tx=0x7f52e802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.163+0000 7f52f6b4f640 1 -- 172.21.15.72:0/2715922427 shutdown_connections 2024-04-25T13:12:28.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.163+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/2715922427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0102990 0x7f52f01092b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.163+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/2715922427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52f0101fd0 0x7f52f0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.163+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/2715922427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52f0100e80 0x7f52f01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.163+0000 7f52f6b4f640 1 -- 172.21.15.72:0/2715922427 >> 172.21.15.72:0/2715922427 conn(0x7f52f00fc3f0 msgr2=0x7f52f00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.163+0000 7f52f6b4f640 1 -- 172.21.15.72:0/2715922427 shutdown_connections 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.163+0000 7f52f6b4f640 1 -- 172.21.15.72:0/2715922427 wait complete. 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52f6b4f640 1 Processor -- start 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52f6b4f640 1 -- start start 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52f6b4f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52f0100e80 0x7f52f019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52f6b4f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0101fd0 0x7f52f019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52f6b4f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52f0102990 0x7f52f01a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52f6b4f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f52f010b950 con 0x7f52f0101fd0 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52f6b4f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52f010b7d0 con 0x7f52f0102990 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52f6b4f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52f010bad0 con 0x7f52f0100e80 2024-04-25T13:12:28.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52effff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0101fd0 0x7f52f019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52effff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0101fd0 0x7f52f019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34480/0 (socket says 172.21.15.72:34480) 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52effff640 1 -- 172.21.15.72:0/522344568 learned_addr learned my addr 172.21.15.72:0/522344568 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52effff640 1 -- 172.21.15.72:0/522344568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52f0100e80 msgr2=0x7f52f019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52effff640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52f0100e80 0x7f52f019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52effff640 1 -- 172.21.15.72:0/522344568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52f0102990 msgr2=0x7f52f01a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52effff640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52f0102990 0x7f52f01a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52effff640 1 -- 172.21.15.72:0/522344568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f01a1ff0 con 0x7f52f0101fd0 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52effff640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0101fd0 0x7f52f019a910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b450 tx=0x7f52dc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52edffb640 1 -- 172.21.15.72:0/522344568 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc015020 con 0x7f52f0101fd0 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52edffb640 1 -- 172.21.15.72:0/522344568 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52dc011d40 con 0x7f52f0101fd0 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f01a22e0 con 0x7f52f0101fd0 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.164+0000 7f52edffb640 1 -- 172.21.15.72:0/522344568 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00f340 con 0x7f52f0101fd0 2024-04-25T13:12:28.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.165+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f01a28b0 con 0x7f52f0101fd0 2024-04-25T13:12:28.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.166+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f0108190 con 0x7f52f0101fd0 2024-04-25T13:12:28.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.167+0000 7f52edffb640 1 -- 172.21.15.72:0/522344568 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f52dc018020 con 0x7f52f0101fd0 2024-04-25T13:12:28.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.167+0000 7f52edffb640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f52e00a3ca0 0x7f52e00a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.167+0000 7f52f48eb640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f52e00a3ca0 0x7f52e00a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:28.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.167+0000 7f52edffb640 1 -- 172.21.15.72:0/522344568 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f52dc0cffb0 con 0x7f52f0101fd0 2024-04-25T13:12:28.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.168+0000 7f52f48eb640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f52e00a3ca0 0x7f52e00a6160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f52e400a990 tx=0x7f52e4005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:28.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.172+0000 7f52edffb640 1 -- 172.21.15.72:0/522344568 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52dc01f020 con 0x7f52f0101fd0 2024-04-25T13:12:28.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.351+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f52f002d7e0 con 0x7f52f0101fd0 2024-04-25T13:12:28.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.351+0000 7f52edffb640 1 -- 172.21.15.72:0/522344568 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f52dc00d040 con 0x7f52f0101fd0 2024-04-25T13:12:28.351 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.353+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f52e00a3ca0 msgr2=0x7f52e00a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.353+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f52e00a3ca0 0x7f52e00a6160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f52e400a990 tx=0x7f52e4005c70 comp rx=0 tx=0).stop 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.353+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0101fd0 msgr2=0x7f52f019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.353+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0101fd0 0x7f52f019a910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b450 tx=0x7f52dc00b910 comp rx=0 tx=0).stop 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.354+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 shutdown_connections 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.354+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f52e00a3ca0 0x7f52e00a6160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.354+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52f0102990 0x7f52f01a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.354+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f0101fd0 0x7f52f019a910 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.354+0000 7f52f6b4f640 1 --2- 172.21.15.72:0/522344568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52f0100e80 0x7f52f019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.354+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 >> 172.21.15.72:0/522344568 conn(0x7f52f00fc3f0 msgr2=0x7f52f0105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:28.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.354+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 shutdown_connections 2024-04-25T13:12:28.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.354+0000 7f52f6b4f640 1 -- 172.21.15.72:0/522344568 wait complete. 2024-04-25T13:12:28.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:28 smithi072 ceph-mon[29199]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:28.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:28 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2046159597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:28.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:28 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/455594288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:28.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:28 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/391802862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:28.365 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:12:28.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 -- 172.21.15.72:0/778478742 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50100e90 msgr2=0x7f4b501012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 --2- 172.21.15.72:0/778478742 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50100e90 0x7f4b501012b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009900 tx=0x7f4b4002a410 comp rx=0 tx=0).stop 2024-04-25T13:12:28.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 -- 172.21.15.72:0/778478742 shutdown_connections 2024-04-25T13:12:28.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 --2- 172.21.15.72:0/778478742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b501029a0 0x7f4b501092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 --2- 172.21.15.72:0/778478742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50101fe0 0x7f4b50102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 --2- 172.21.15.72:0/778478742 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50100e90 0x7f4b501012b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 -- 172.21.15.72:0/778478742 >> 172.21.15.72:0/778478742 conn(0x7f4b500fc440 msgr2=0x7f4b500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:28.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 -- 172.21.15.72:0/778478742 shutdown_connections 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 -- 172.21.15.72:0/778478742 wait complete. 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.462+0000 7f4b56115640 1 Processor -- start 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b56115640 1 -- start start 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b56115640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50100e90 0x7f4b50198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b56115640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50101fe0 0x7f4b50198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b56115640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b501029a0 0x7f4b5019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b56115640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010c9e0 con 0x7f4b50101fe0 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b56115640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010c860 con 0x7f4b501029a0 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b56115640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010cb60 con 0x7f4b50100e90 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50101fe0 0x7f4b50198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50101fe0 0x7f4b50198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34498/0 (socket says 172.21.15.72:34498) 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4effd640 1 -- 172.21.15.72:0/246619010 learned_addr learned my addr 172.21.15.72:0/246619010 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:28.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4effd640 1 -- 172.21.15.72:0/246619010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50100e90 msgr2=0x7f4b50198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4effd640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50100e90 0x7f4b50198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4effd640 1 -- 172.21.15.72:0/246619010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b501029a0 msgr2=0x7f4b5019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4effd640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b501029a0 0x7f4b5019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4effd640 1 -- 172.21.15.72:0/246619010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5019fd60 con 0x7f4b50101fe0 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4effd640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50101fe0 0x7f4b50198680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400c970 tx=0x7f4b4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4cff9640 1 -- 172.21.15.72:0/246619010 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44007c10 con 0x7f4b50101fe0 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4cff9640 1 -- 172.21.15.72:0/246619010 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b44007db0 con 0x7f4b50101fe0 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5019fff0 con 0x7f4b50101fe0 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b4cff9640 1 -- 172.21.15.72:0/246619010 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4400f870 con 0x7f4b50101fe0 2024-04-25T13:12:28.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.463+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501a0510 con 0x7f4b50101fe0 2024-04-25T13:12:28.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.464+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b501a07d0 con 0x7f4b50101fe0 2024-04-25T13:12:28.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.465+0000 7f4b4cff9640 1 -- 172.21.15.72:0/246619010 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4b4400fa10 con 0x7f4b50101fe0 2024-04-25T13:12:28.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.466+0000 7f4b4cff9640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4b380a3c40 0x7f4b380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.466+0000 7f4b4f7fe640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4b380a3c40 0x7f4b380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:28.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.466+0000 7f4b4cff9640 1 -- 172.21.15.72:0/246619010 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f4b440d0c70 con 0x7f4b50101fe0 2024-04-25T13:12:28.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.466+0000 7f4b4f7fe640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4b380a3c40 0x7f4b380a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4b400096c0 tx=0x7f4b40000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:28.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.469+0000 7f4b4cff9640 1 -- 172.21.15.72:0/246619010 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b4409d600 con 0x7f4b50101fe0 2024-04-25T13:12:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:28 smithi193 ceph-mon[32743]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:28 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2046159597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:28 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/455594288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:28.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:28 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/391802862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:28.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:28 smithi164 ceph-mon[32418]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:28.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:28 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2046159597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:28.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:28 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/455594288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:28.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:28 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/391802862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:28.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.686+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b501a09a0 con 0x7f4b50101fe0 2024-04-25T13:12:28.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.687+0000 7f4b4cff9640 1 -- 172.21.15.72:0/246619010 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11881 (secure 0 0 0) 0x7f4b4409cf00 con 0x7f4b50101fe0 2024-04-25T13:12:28.687 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:28.687 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:11:04.601639+0000","modified":"2024-04-25T13:11:09.292132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24493,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6839/1444196308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1444196308},{"type":"v1","addr":"172.21.15.164:6839","nonce":1444196308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24478,24493,24509]},"id":1}]} 2024-04-25T13:12:28.687 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 20 2024-04-25T13:12:28.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4b380a3c40 msgr2=0x7f4b380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4b380a3c40 0x7f4b380a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4b400096c0 tx=0x7f4b40000bc0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50101fe0 msgr2=0x7f4b50198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50101fe0 0x7f4b50198680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400c970 tx=0x7f4b4400ce30 comp rx=0 tx=0).stop 2024-04-25T13:12:28.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 shutdown_connections 2024-04-25T13:12:28.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4b380a3c40 0x7f4b380a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b501029a0 0x7f4b5019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b50101fe0 0x7f4b50198680 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 --2- 172.21.15.72:0/246619010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50100e90 0x7f4b50198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 >> 172.21.15.72:0/246619010 conn(0x7f4b500fc440 msgr2=0x7f4b500fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:28.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 shutdown_connections 2024-04-25T13:12:28.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.689+0000 7f4b56115640 1 -- 172.21.15.72:0/246619010 wait complete. 2024-04-25T13:12:28.701 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:12:28.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 -- 172.21.15.72:0/3059809880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91441053f0 msgr2=0x7f91441077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 --2- 172.21.15.72:0/3059809880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91441053f0 0x7f91441077e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9138009900 tx=0x7f913802a410 comp rx=0 tx=0).stop 2024-04-25T13:12:28.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 -- 172.21.15.72:0/3059809880 shutdown_connections 2024-04-25T13:12:28.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 --2- 172.21.15.72:0/3059809880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9144107d20 0x7f914410a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 --2- 172.21.15.72:0/3059809880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91441053f0 0x7f91441077e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 --2- 172.21.15.72:0/3059809880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9144102ac0 0x7f9144104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 -- 172.21.15.72:0/3059809880 >> 172.21.15.72:0/3059809880 conn(0x7f91440fc450 msgr2=0x7f91440fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 -- 172.21.15.72:0/3059809880 shutdown_connections 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 -- 172.21.15.72:0/3059809880 wait complete. 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.798+0000 7f914907c640 1 Processor -- start 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914907c640 1 -- start start 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914907c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9144102ac0 0x7f9144198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914907c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91441053f0 0x7f91441987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914907c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9144107d20 0x7f914419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914907c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f914410db20 con 0x7f9144107d20 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914907c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f914410d9a0 con 0x7f91441053f0 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914907c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f914410dca0 con 0x7f9144102ac0 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914359e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9144107d20 0x7f914419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914359e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9144107d20 0x7f914419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34518/0 (socket says 172.21.15.72:34518) 2024-04-25T13:12:28.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914359e640 1 -- 172.21.15.72:0/3601099727 learned_addr learned my addr 172.21.15.72:0/3601099727 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914259c640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91441053f0 0x7f91441987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f9142d9d640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9144102ac0 0x7f9144198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914359e640 1 -- 172.21.15.72:0/3601099727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9144102ac0 msgr2=0x7f9144198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914359e640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9144102ac0 0x7f9144198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914359e640 1 -- 172.21.15.72:0/3601099727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91441053f0 msgr2=0x7f91441987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914359e640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91441053f0 0x7f91441987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914359e640 1 -- 172.21.15.72:0/3601099727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914419fea0 con 0x7f9144107d20 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914359e640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9144107d20 0x7f914419f830 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f913c00eaa0 tx=0x7f913c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f912bfff640 1 -- 172.21.15.72:0/3601099727 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c00cdb0 con 0x7f9144107d20 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f912bfff640 1 -- 172.21.15.72:0/3601099727 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f913c005020 con 0x7f9144107d20 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91441a0190 con 0x7f9144107d20 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f912bfff640 1 -- 172.21.15.72:0/3601099727 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c0116d0 con 0x7f9144107d20 2024-04-25T13:12:28.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.799+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91441a06e0 con 0x7f9144107d20 2024-04-25T13:12:28.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.801+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91440fdf40 con 0x7f9144107d20 2024-04-25T13:12:28.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.801+0000 7f912bfff640 1 -- 172.21.15.72:0/3601099727 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f913c0051d0 con 0x7f9144107d20 2024-04-25T13:12:28.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.802+0000 7f912bfff640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f912c0a3c40 0x7f912c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:28.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.802+0000 7f9142d9d640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f912c0a3c40 0x7f912c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:28.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.802+0000 7f912bfff640 1 -- 172.21.15.72:0/3601099727 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f913c0d0850 con 0x7f9144107d20 2024-04-25T13:12:28.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.802+0000 7f9142d9d640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f912c0a3c40 0x7f912c0a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9134009890 tx=0x7f9134008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:28.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.807+0000 7f912bfff640 1 -- 172.21.15.72:0/3601099727 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f913c016030 con 0x7f9144107d20 2024-04-25T13:12:28.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.985+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9144062b50 con 0x7f9144107d20 2024-04-25T13:12:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.987+0000 7f912bfff640 1 -- 172.21.15.72:0/3601099727 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22740 (secure 0 0 0) 0x7f913c09d0c0 con 0x7f9144107d20 2024-04-25T13:12:28.987 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:28.987 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:12:08.140624+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:12:28.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f912c0a3c40 msgr2=0x7f912c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f912c0a3c40 0x7f912c0a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9134009890 tx=0x7f9134008040 comp rx=0 tx=0).stop 2024-04-25T13:12:28.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9144107d20 msgr2=0x7f914419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:28.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9144107d20 0x7f914419f830 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f913c00eaa0 tx=0x7f913c00ef60 comp rx=0 tx=0).stop 2024-04-25T13:12:28.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 shutdown_connections 2024-04-25T13:12:28.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f912c0a3c40 0x7f912c0a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9144107d20 0x7f914419f830 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91441053f0 0x7f91441987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 --2- 172.21.15.72:0/3601099727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9144102ac0 0x7f9144198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:28.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 >> 172.21.15.72:0/3601099727 conn(0x7f91440fc450 msgr2=0x7f91441034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:28.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 shutdown_connections 2024-04-25T13:12:28.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:28.989+0000 7f914907c640 1 -- 172.21.15.72:0/3601099727 wait complete. 2024-04-25T13:12:29.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:06:33.150709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:11:02.352089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:11:03.565033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:12:29.001 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:12:29.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.102+0000 7f8481bbb640 1 -- 172.21.15.72:0/3679995746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 msgr2=0x7f847c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.102+0000 7f8481bbb640 1 --2- 172.21.15.72:0/3679995746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 0x7f847c10a3e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8474009ca0 tx=0x7f847402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.102+0000 7f8481bbb640 1 -- 172.21.15.72:0/3679995746 shutdown_connections 2024-04-25T13:12:29.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.102+0000 7f8481bbb640 1 --2- 172.21.15.72:0/3679995746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 0x7f847c10a3e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.102+0000 7f8481bbb640 1 --2- 172.21.15.72:0/3679995746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f847c103190 0x7f847c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.102+0000 7f8481bbb640 1 --2- 172.21.15.72:0/3679995746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f847c101fa0 0x7f847c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.102+0000 7f8481bbb640 1 -- 172.21.15.72:0/3679995746 >> 172.21.15.72:0/3679995746 conn(0x7f847c0fd530 msgr2=0x7f847c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:29.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.102+0000 7f8481bbb640 1 -- 172.21.15.72:0/3679995746 shutdown_connections 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.102+0000 7f8481bbb640 1 -- 172.21.15.72:0/3679995746 wait complete. 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f8481bbb640 1 Processor -- start 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f8481bbb640 1 -- start start 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f8481bbb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f847c101fa0 0x7f847c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f8481bbb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f847c103190 0x7f847c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f8481bbb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 0x7f847c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f8481bbb640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f847c10db20 con 0x7f847c103b50 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f8481bbb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f847c10d9a0 con 0x7f847c101fa0 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f8481bbb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f847c10dca0 con 0x7f847c103190 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f847bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 0x7f847c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f847bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 0x7f847c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34532/0 (socket says 172.21.15.72:34532) 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f847bfff640 1 -- 172.21.15.72:0/2765613382 learned_addr learned my addr 172.21.15.72:0/2765613382 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:29.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f847b7fe640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f847c101fa0 0x7f847c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.103+0000 7f847bfff640 1 -- 172.21.15.72:0/2765613382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f847c103190 msgr2=0x7f847c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f8473fff640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f847c103190 0x7f847c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f847bfff640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f847c103190 0x7f847c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f847bfff640 1 -- 172.21.15.72:0/2765613382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f847c101fa0 msgr2=0x7f847c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f847bfff640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f847c101fa0 0x7f847c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f847bfff640 1 -- 172.21.15.72:0/2765613382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847c19fdb0 con 0x7f847c103b50 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f847bfff640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 0x7f847c19f740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8474031a20 tx=0x7f8474003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f84797fa640 1 -- 172.21.15.72:0/2765613382 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474004240 con 0x7f847c103b50 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f84797fa640 1 -- 172.21.15.72:0/2765613382 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8474003e10 con 0x7f847c103b50 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847c1a0040 con 0x7f847c103b50 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f84797fa640 1 -- 172.21.15.72:0/2765613382 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84740305c0 con 0x7f847c103b50 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.104+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847c1a0610 con 0x7f847c103b50 2024-04-25T13:12:29.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.105+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847c1092c0 con 0x7f847c103b50 2024-04-25T13:12:29.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.106+0000 7f84797fa640 1 -- 172.21.15.72:0/2765613382 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f847402f660 con 0x7f847c103b50 2024-04-25T13:12:29.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.106+0000 7f84797fa640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f84680a3c40 0x7f84680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.106+0000 7f847b7fe640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f84680a3c40 0x7f84680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.106+0000 7f84797fa640 1 -- 172.21.15.72:0/2765613382 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f84740f1e20 con 0x7f847c103b50 2024-04-25T13:12:29.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.107+0000 7f847b7fe640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f84680a3c40 0x7f84680a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f846c0058c0 tx=0x7f846c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:29.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.111+0000 7f84797fa640 1 -- 172.21.15.72:0/2765613382 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84740be7a0 con 0x7f847c103b50 2024-04-25T13:12:29.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:29 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/522344568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:29.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:29 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/246619010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:29.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:29 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3601099727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:29.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.291+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f847c062b50 con 0x7f847c103b50 2024-04-25T13:12:29.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.291+0000 7f84797fa640 1 -- 172.21.15.72:0/2765613382 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f84740be0a0 con 0x7f847c103b50 2024-04-25T13:12:29.291 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-25T13:12:29.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f84680a3c40 msgr2=0x7f84680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f84680a3c40 0x7f84680a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f846c0058c0 tx=0x7f846c005850 comp rx=0 tx=0).stop 2024-04-25T13:12:29.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 msgr2=0x7f847c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 0x7f847c19f740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8474031a20 tx=0x7f8474003c20 comp rx=0 tx=0).stop 2024-04-25T13:12:29.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 shutdown_connections 2024-04-25T13:12:29.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f84680a3c40 0x7f84680a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c103b50 0x7f847c19f740 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f847c103190 0x7f847c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 --2- 172.21.15.72:0/2765613382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f847c101fa0 0x7f847c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 >> 172.21.15.72:0/2765613382 conn(0x7f847c0fd530 msgr2=0x7f847c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:29.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 shutdown_connections 2024-04-25T13:12:29.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.293+0000 7f8481bbb640 1 -- 172.21.15.72:0/2765613382 wait complete. 2024-04-25T13:12:29.304 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:12:29.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 -- 172.21.15.72:0/3127826376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c101fd0 msgr2=0x7fe22c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 --2- 172.21.15.72:0/3127826376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c101fd0 0x7fe22c102450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009900 tx=0x7fe21c02a410 comp rx=0 tx=0).stop 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 -- 172.21.15.72:0/3127826376 shutdown_connections 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 --2- 172.21.15.72:0/3127826376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe22c102990 0x7fe22c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 --2- 172.21.15.72:0/3127826376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c101fd0 0x7fe22c102450 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 --2- 172.21.15.72:0/3127826376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22c100e80 0x7fe22c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 -- 172.21.15.72:0/3127826376 >> 172.21.15.72:0/3127826376 conn(0x7fe22c0fc3f0 msgr2=0x7fe22c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 -- 172.21.15.72:0/3127826376 shutdown_connections 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 -- 172.21.15.72:0/3127826376 wait complete. 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 Processor -- start 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 -- start start 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.405+0000 7fe233a8a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe22c100e80 0x7fe22c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe233a8a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22c101fd0 0x7fe22c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe233a8a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c102990 0x7fe22c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe233a8a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c10c9d0 con 0x7fe22c102990 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe233a8a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c10c850 con 0x7fe22c101fd0 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe233a8a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c10cb50 con 0x7fe22c100e80 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe232027640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c102990 0x7fe22c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe232027640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c102990 0x7fe22c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34548/0 (socket says 172.21.15.72:34548) 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe232027640 1 -- 172.21.15.72:0/683685550 learned_addr learned my addr 172.21.15.72:0/683685550 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe231826640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe22c100e80 0x7fe22c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe231025640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22c101fd0 0x7fe22c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe232027640 1 -- 172.21.15.72:0/683685550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe22c100e80 msgr2=0x7fe22c197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe232027640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe22c100e80 0x7fe22c197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe232027640 1 -- 172.21.15.72:0/683685550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22c101fd0 msgr2=0x7fe22c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe232027640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22c101fd0 0x7fe22c1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe232027640 1 -- 172.21.15.72:0/683685550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c19ec90 con 0x7fe22c102990 2024-04-25T13:12:29.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe232027640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c102990 0x7fe22c19e620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe22400eaa0 tx=0x7fe22400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:29.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe222ffd640 1 -- 172.21.15.72:0/683685550 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22400cdb0 con 0x7fe22c102990 2024-04-25T13:12:29.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c19ef80 con 0x7fe22c102990 2024-04-25T13:12:29.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe222ffd640 1 -- 172.21.15.72:0/683685550 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe224005020 con 0x7fe22c102990 2024-04-25T13:12:29.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe222ffd640 1 -- 172.21.15.72:0/683685550 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2240116d0 con 0x7fe22c102990 2024-04-25T13:12:29.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.406+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c19f440 con 0x7fe22c102990 2024-04-25T13:12:29.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.407+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c1081e0 con 0x7fe22c102990 2024-04-25T13:12:29.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.408+0000 7fe222ffd640 1 -- 172.21.15.72:0/683685550 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe2240054a0 con 0x7fe22c102990 2024-04-25T13:12:29.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.409+0000 7fe222ffd640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe2180a3c40 0x7fe2180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.409+0000 7fe231826640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe2180a3c40 0x7fe2180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.409+0000 7fe222ffd640 1 -- 172.21.15.72:0/683685550 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fe2240d0720 con 0x7fe22c102990 2024-04-25T13:12:29.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.409+0000 7fe231826640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe2180a3c40 0x7fe2180a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe22800bbe0 tx=0x7fe22800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:29.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.413+0000 7fe222ffd640 1 -- 172.21.15.72:0/683685550 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe224016030 con 0x7fe22c102990 2024-04-25T13:12:29.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:29 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/522344568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:29.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:29 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/246619010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:29.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:29 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3601099727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:29 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/522344568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:29 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/246619010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:29 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3601099727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:29.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.590+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe22c02d770 con 0x7fe22c102990 2024-04-25T13:12:29.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.591+0000 7fe222ffd640 1 -- 172.21.15.72:0/683685550 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fe224085e20 con 0x7fe22c102990 2024-04-25T13:12:29.591 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-25T13:12:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe2180a3c40 msgr2=0x7fe2180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe2180a3c40 0x7fe2180a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe22800bbe0 tx=0x7fe22800b390 comp rx=0 tx=0).stop 2024-04-25T13:12:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c102990 msgr2=0x7fe22c19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c102990 0x7fe22c19e620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe22400eaa0 tx=0x7fe22400ef60 comp rx=0 tx=0).stop 2024-04-25T13:12:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 shutdown_connections 2024-04-25T13:12:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe2180a3c40 0x7fe2180a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe22c102990 0x7fe22c19e620 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22c101fd0 0x7fe22c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 --2- 172.21.15.72:0/683685550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe22c100e80 0x7fe22c197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 >> 172.21.15.72:0/683685550 conn(0x7fe22c0fc3f0 msgr2=0x7fe22c1053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:29.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 shutdown_connections 2024-04-25T13:12:29.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.593+0000 7fe233a8a640 1 -- 172.21.15.72:0/683685550 wait complete. 2024-04-25T13:12:29.604 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:12:29.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.705+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1329385932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8105340 msgr2=0x7fd4b8107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.705+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1329385932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8105340 0x7fd4b8107730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009900 tx=0x7fd4ac02a410 comp rx=0 tx=0).stop 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.705+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1329385932 shutdown_connections 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.705+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1329385932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4b8107c70 0x7fd4b810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.705+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1329385932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8105340 0x7fd4b8107730 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.705+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1329385932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b8102a10 0x7fd4b8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.705+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1329385932 >> 172.21.15.72:0/1329385932 conn(0x7fd4b80fc400 msgr2=0x7fd4b80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.705+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1329385932 shutdown_connections 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.705+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1329385932 wait complete. 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4bc8e8640 1 Processor -- start 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4bc8e8640 1 -- start start 2024-04-25T13:12:29.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4bc8e8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4b8102a10 0x7fd4b8198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4bc8e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b8105340 0x7fd4b8198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4bc8e8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8107c70 0x7fd4b819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4bc8e8640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b810daa0 con 0x7fd4b8107c70 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4bc8e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b810d920 con 0x7fd4b8105340 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4bc8e8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b810dc20 con 0x7fd4b8102a10 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b6d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8107c70 0x7fd4b819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b6d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8107c70 0x7fd4b819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34570/0 (socket says 172.21.15.72:34570) 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b6d9d640 1 -- 172.21.15.72:0/1117266523 learned_addr learned my addr 172.21.15.72:0/1117266523 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b659c640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4b8102a10 0x7fd4b8198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b5d9b640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b8105340 0x7fd4b8198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b6d9d640 1 -- 172.21.15.72:0/1117266523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4b8102a10 msgr2=0x7fd4b8198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b6d9d640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4b8102a10 0x7fd4b8198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b6d9d640 1 -- 172.21.15.72:0/1117266523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b8105340 msgr2=0x7fd4b8198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b6d9d640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b8105340 0x7fd4b8198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b6d9d640 1 -- 172.21.15.72:0/1117266523 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b819fe10 con 0x7fd4b8107c70 2024-04-25T13:12:29.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.706+0000 7fd4b5d9b640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b8105340 0x7fd4b8198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:29.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.707+0000 7fd4b6d9d640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8107c70 0x7fd4b819f7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009fd0 tx=0x7fd4b000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:29.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.707+0000 7fd4a77fe640 1 -- 172.21.15.72:0/1117266523 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0017070 con 0x7fd4b8107c70 2024-04-25T13:12:29.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.707+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b81a0100 con 0x7fd4b8107c70 2024-04-25T13:12:29.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.707+0000 7fd4a77fe640 1 -- 172.21.15.72:0/1117266523 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b0003e20 con 0x7fd4b8107c70 2024-04-25T13:12:29.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.707+0000 7fd4a77fe640 1 -- 172.21.15.72:0/1117266523 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0010460 con 0x7fd4b8107c70 2024-04-25T13:12:29.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.707+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b81a05d0 con 0x7fd4b8107c70 2024-04-25T13:12:29.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.708+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b80fdef0 con 0x7fd4b8107c70 2024-04-25T13:12:29.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.709+0000 7fd4a77fe640 1 -- 172.21.15.72:0/1117266523 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd4b0003970 con 0x7fd4b8107c70 2024-04-25T13:12:29.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.709+0000 7fd4a77fe640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4a00a3c40 0x7fd4a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:29.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.710+0000 7fd4b659c640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4a00a3c40 0x7fd4a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:29.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.710+0000 7fd4a77fe640 1 -- 172.21.15.72:0/1117266523 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fd4b00d0360 con 0x7fd4b8107c70 2024-04-25T13:12:29.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.710+0000 7fd4b659c640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4a00a3c40 0x7fd4a00a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800bbe0 tx=0x7fd4a800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:29.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.714+0000 7fd4a77fe640 1 -- 172.21.15.72:0/1117266523 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4b009cbd0 con 0x7fd4b8107c70 2024-04-25T13:12:29.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.893+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd4b8062b50 con 0x7fd4b8107c70 2024-04-25T13:12:29.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.894+0000 7fd4a77fe640 1 -- 172.21.15.72:0/1117266523 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fd4b009c4d0 con 0x7fd4b8107c70 2024-04-25T13:12:29.894 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-25T13:12:29.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4a00a3c40 msgr2=0x7fd4a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4a00a3c40 0x7fd4a00a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800bbe0 tx=0x7fd4a800b390 comp rx=0 tx=0).stop 2024-04-25T13:12:29.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8107c70 msgr2=0x7fd4b819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:29.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8107c70 0x7fd4b819f7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009fd0 tx=0x7fd4b000b260 comp rx=0 tx=0).stop 2024-04-25T13:12:29.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 shutdown_connections 2024-04-25T13:12:29.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd4a00a3c40 0x7fd4a00a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4b8107c70 0x7fd4b819f7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b8105340 0x7fd4b8198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 --2- 172.21.15.72:0/1117266523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4b8102a10 0x7fd4b8198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:29.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 >> 172.21.15.72:0/1117266523 conn(0x7fd4b80fc400 msgr2=0x7fd4b8103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:29.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 shutdown_connections 2024-04-25T13:12:29.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:29.896+0000 7fd4bc8e8640 1 -- 172.21.15.72:0/1117266523 wait complete. 2024-04-25T13:12:29.908 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.006+0000 7fe9ac95d640 1 -- 172.21.15.72:0/2538682426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a8101b90 msgr2=0x7fe9a8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.006+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/2538682426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a8101b90 0x7fe9a8102010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009ca0 tx=0x7fe9a002a950 comp rx=0 tx=0).stop 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 -- 172.21.15.72:0/2538682426 shutdown_connections 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/2538682426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a8101b90 0x7fe9a8102010 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/2538682426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9a81011f0 0x7fe9a8101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/2538682426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9a8100890 0x7fe9a8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 -- 172.21.15.72:0/2538682426 >> 172.21.15.72:0/2538682426 conn(0x7fe9a80fc430 msgr2=0x7fe9a80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 -- 172.21.15.72:0/2538682426 shutdown_connections 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 -- 172.21.15.72:0/2538682426 wait complete. 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 Processor -- start 2024-04-25T13:12:30.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 -- start start 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9a8100890 0x7fe9a819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a81011f0 0x7fe9a819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9a8101b90 0x7fe9a81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a810fce0 con 0x7fe9a81011f0 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a810fb60 con 0x7fe9a8100890 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.007+0000 7fe9ac95d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a810fe60 con 0x7fe9a8101b90 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a5d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a81011f0 0x7fe9a819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a5d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a81011f0 0x7fe9a819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34588/0 (socket says 172.21.15.72:34588) 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a5d9b640 1 -- 172.21.15.72:0/4160508952 learned_addr learned my addr 172.21.15.72:0/4160508952 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a659c640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9a8100890 0x7fe9a819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a5d9b640 1 -- 172.21.15.72:0/4160508952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9a8101b90 msgr2=0x7fe9a81a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a5d9b640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9a8101b90 0x7fe9a81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a5d9b640 1 -- 172.21.15.72:0/4160508952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9a8100890 msgr2=0x7fe9a819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a5d9b640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9a8100890 0x7fe9a819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a5d9b640 1 -- 172.21.15.72:0/4160508952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a81a1ea0 con 0x7fe9a81011f0 2024-04-25T13:12:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a659c640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9a8100890 0x7fe9a819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9a5d9b640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a81011f0 0x7fe9a819a7c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c920 tx=0x7fe99c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9977fe640 1 -- 172.21.15.72:0/4160508952 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c007c70 con 0x7fe9a81011f0 2024-04-25T13:12:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9977fe640 1 -- 172.21.15.72:0/4160508952 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99c007e10 con 0x7fe9a81011f0 2024-04-25T13:12:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a81a2190 con 0x7fe9a81011f0 2024-04-25T13:12:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9977fe640 1 -- 172.21.15.72:0/4160508952 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c018820 con 0x7fe9a81011f0 2024-04-25T13:12:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.008+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a81a26e0 con 0x7fe9a81011f0 2024-04-25T13:12:30.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.009+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a810b6e0 con 0x7fe9a81011f0 2024-04-25T13:12:30.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.010+0000 7fe9977fe640 1 -- 172.21.15.72:0/4160508952 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe99c00f450 con 0x7fe9a81011f0 2024-04-25T13:12:30.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.011+0000 7fe9977fe640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe9900a3c40 0x7fe9900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.011+0000 7fe9a659c640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe9900a3c40 0x7fe9900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.011+0000 7fe9977fe640 1 -- 172.21.15.72:0/4160508952 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fe99c0d0760 con 0x7fe9a81011f0 2024-04-25T13:12:30.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.011+0000 7fe9a659c640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe9900a3c40 0x7fe9900a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe99800baa0 tx=0x7fe998005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:30.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.015+0000 7fe9977fe640 1 -- 172.21.15.72:0/4160508952 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe99c014030 con 0x7fe9a81011f0 2024-04-25T13:12:30.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.232+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9a802d800 con 0x7fe9a81011f0 2024-04-25T13:12:30.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.233+0000 7fe9977fe640 1 -- 172.21.15.72:0/4160508952 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11881 (secure 0 0 0) 0x7fe99c09cfd0 con 0x7fe9a81011f0 2024-04-25T13:12:30.233 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:11:04.601639+0000","modified":"2024-04-25T13:11:09.292132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24493,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6839/1444196308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1444196308},{"type":"v1","addr":"172.21.15.164:6839","nonce":1444196308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24478,24493,24509]},"id":1}]} 2024-04-25T13:12:30.233 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 20 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe9900a3c40 msgr2=0x7fe9900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe9900a3c40 0x7fe9900a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe99800baa0 tx=0x7fe998005d90 comp rx=0 tx=0).stop 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a81011f0 msgr2=0x7fe9a819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a81011f0 0x7fe9a819a7c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c920 tx=0x7fe99c00cde0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 shutdown_connections 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fe9900a3c40 0x7fe9900a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9a8101b90 0x7fe9a81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9a81011f0 0x7fe9a819a7c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 --2- 172.21.15.72:0/4160508952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9a8100890 0x7fe9a819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 >> 172.21.15.72:0/4160508952 conn(0x7fe9a80fc430 msgr2=0x7fe9a80fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 shutdown_connections 2024-04-25T13:12:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.236+0000 7fe9ac95d640 1 -- 172.21.15.72:0/4160508952 wait complete. 2024-04-25T13:12:30.247 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:12:30.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.344+0000 7ff633bbb640 1 -- 172.21.15.72:0/2444618754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c100f10 msgr2=0x7ff62c101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.344+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2444618754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c100f10 0x7ff62c101330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff628009900 tx=0x7ff62802a410 comp rx=0 tx=0).stop 2024-04-25T13:12:30.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.344+0000 7ff633bbb640 1 -- 172.21.15.72:0/2444618754 shutdown_connections 2024-04-25T13:12:30.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.344+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2444618754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c102a20 0x7ff62c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.344+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2444618754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62c102060 0x7ff62c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.344+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2444618754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c100f10 0x7ff62c101330 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.344+0000 7ff633bbb640 1 -- 172.21.15.72:0/2444618754 >> 172.21.15.72:0/2444618754 conn(0x7ff62c0fc4a0 msgr2=0x7ff62c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.344+0000 7ff633bbb640 1 -- 172.21.15.72:0/2444618754 shutdown_connections 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.344+0000 7ff633bbb640 1 -- 172.21.15.72:0/2444618754 wait complete. 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff633bbb640 1 Processor -- start 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff633bbb640 1 -- start start 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff633bbb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62c100f10 0x7ff62c10b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff633bbb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c102060 0x7ff62c109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff633bbb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c102a20 0x7ff62c10a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff633bbb640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c10db30 con 0x7ff62c102060 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff633bbb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c10d9b0 con 0x7ff62c100f10 2024-04-25T13:12:30.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff633bbb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c10dcb0 con 0x7ff62c102a20 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff631156640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c102060 0x7ff62c109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff631156640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c102060 0x7ff62c109ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34614/0 (socket says 172.21.15.72:34614) 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff631156640 1 -- 172.21.15.72:0/2085106592 learned_addr learned my addr 172.21.15.72:0/2085106592 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff631957640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62c100f10 0x7ff62c10b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff632158640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c102a20 0x7ff62c10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff631156640 1 -- 172.21.15.72:0/2085106592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c102a20 msgr2=0x7ff62c10a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff631156640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c102a20 0x7ff62c10a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff631156640 1 -- 172.21.15.72:0/2085106592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62c100f10 msgr2=0x7ff62c10b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.345+0000 7ff631156640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62c100f10 0x7ff62c10b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.346+0000 7ff631156640 1 -- 172.21.15.72:0/2085106592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c10acc0 con 0x7ff62c102060 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.346+0000 7ff631957640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62c100f10 0x7ff62c10b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.346+0000 7ff632158640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c102a20 0x7ff62c10a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:30.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.346+0000 7ff631156640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c102060 0x7ff62c109ef0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c970 tx=0x7ff61c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:30.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.346+0000 7ff622ffd640 1 -- 172.21.15.72:0/2085106592 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c004220 con 0x7ff62c102060 2024-04-25T13:12:30.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.346+0000 7ff622ffd640 1 -- 172.21.15.72:0/2085106592 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff61c0043c0 con 0x7ff62c102060 2024-04-25T13:12:30.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.346+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c1a2310 con 0x7ff62c102060 2024-04-25T13:12:30.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.346+0000 7ff622ffd640 1 -- 172.21.15.72:0/2085106592 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c00f450 con 0x7ff62c102060 2024-04-25T13:12:30.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.346+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c1a28b0 con 0x7ff62c102060 2024-04-25T13:12:30.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.347+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62c108270 con 0x7ff62c102060 2024-04-25T13:12:30.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.348+0000 7ff622ffd640 1 -- 172.21.15.72:0/2085106592 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff61c003610 con 0x7ff62c102060 2024-04-25T13:12:30.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.348+0000 7ff622ffd640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6180a3cc0 0x7ff6180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.348+0000 7ff631957640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6180a3cc0 0x7ff6180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.349+0000 7ff622ffd640 1 -- 172.21.15.72:0/2085106592 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff61c0d0450 con 0x7ff62c102060 2024-04-25T13:12:30.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.349+0000 7ff631957640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6180a3cc0 0x7ff6180a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff628000c00 tx=0x7ff6280314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:30.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.353+0000 7ff622ffd640 1 -- 172.21.15.72:0/2085106592 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff61c014030 con 0x7ff62c102060 2024-04-25T13:12:30.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:30 smithi193 ceph-mon[32743]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:30.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:30 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2765613382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:30.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:30 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/683685550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:30.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:30 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1117266523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:30.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:30 smithi072 ceph-mon[29199]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:30 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2765613382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:30 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/683685550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:30.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:30 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1117266523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:30.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.536+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff62c02d810 con 0x7ff62c102060 2024-04-25T13:12:30.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.538+0000 7ff622ffd640 1 -- 172.21.15.72:0/2085106592 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22740 (secure 0 0 0) 0x7ff61c09ccc0 con 0x7ff62c102060 2024-04-25T13:12:30.538 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:30.538 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:12:08.140624+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:12:30.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6180a3cc0 msgr2=0x7ff6180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6180a3cc0 0x7ff6180a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff628000c00 tx=0x7ff6280314a0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c102060 msgr2=0x7ff62c109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c102060 0x7ff62c109ef0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c970 tx=0x7ff61c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:12:30.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 shutdown_connections 2024-04-25T13:12:30.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff6180a3cc0 0x7ff6180a6180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c102a20 0x7ff62c10a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62c102060 0x7ff62c109ef0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 --2- 172.21.15.72:0/2085106592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62c100f10 0x7ff62c10b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 >> 172.21.15.72:0/2085106592 conn(0x7ff62c0fc4a0 msgr2=0x7ff62c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:30.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 shutdown_connections 2024-04-25T13:12:30.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.540+0000 7ff633bbb640 1 -- 172.21.15.72:0/2085106592 wait complete. 2024-04-25T13:12:30.552 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:12:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:30 smithi164 ceph-mon[32418]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:30 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2765613382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:30 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/683685550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:30.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:30 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1117266523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:30.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.650+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/4004310989 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 msgr2=0x7f0cdc102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.650+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/4004310989 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 0x7f0cdc102460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009900 tx=0x7f0cd002a410 comp rx=0 tx=0).stop 2024-04-25T13:12:30.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.650+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/4004310989 shutdown_connections 2024-04-25T13:12:30.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.650+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/4004310989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cdc1029a0 0x7f0cdc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.650+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/4004310989 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 0x7f0cdc102460 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.650+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/4004310989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cdc100e90 0x7f0cdc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.650+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/4004310989 >> 172.21.15.72:0/4004310989 conn(0x7f0cdc0fc400 msgr2=0x7f0cdc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.650+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/4004310989 shutdown_connections 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.650+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/4004310989 wait complete. 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0ce1e4e640 1 Processor -- start 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0ce1e4e640 1 -- start start 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0ce1e4e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cdc100e90 0x7f0cdc1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0ce1e4e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 0x7f0cdc197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0ce1e4e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cdc1029a0 0x7f0cdc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0ce1e4e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc10c9e0 con 0x7f0cdc101fe0 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0ce1e4e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc10c860 con 0x7f0cdc1029a0 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0ce1e4e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc10cb60 con 0x7f0cdc100e90 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0cdaffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 0x7f0cdc197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0cdaffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 0x7f0cdc197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34626/0 (socket says 172.21.15.72:34626) 2024-04-25T13:12:30.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0cdaffd640 1 -- 172.21.15.72:0/2126085839 learned_addr learned my addr 172.21.15.72:0/2126085839 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0cdaffd640 1 -- 172.21.15.72:0/2126085839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cdc100e90 msgr2=0x7f0cdc1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0cdaffd640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cdc100e90 0x7f0cdc1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0cdaffd640 1 -- 172.21.15.72:0/2126085839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cdc1029a0 msgr2=0x7f0cdc19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0cdaffd640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cdc1029a0 0x7f0cdc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.651+0000 7f0cdaffd640 1 -- 172.21.15.72:0/2126085839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc19ed00 con 0x7f0cdc101fe0 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.652+0000 7f0cdaffd640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 0x7f0cdc197620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0006010 tx=0x7f0cd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.652+0000 7f0cd8ff9640 1 -- 172.21.15.72:0/2126085839 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd003f030 con 0x7f0cdc101fe0 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.652+0000 7f0cd8ff9640 1 -- 172.21.15.72:0/2126085839 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd00318f0 con 0x7f0cdc101fe0 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.652+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc19ef90 con 0x7f0cdc101fe0 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.652+0000 7f0cd8ff9640 1 -- 172.21.15.72:0/2126085839 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0035070 con 0x7f0cdc101fe0 2024-04-25T13:12:30.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.652+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc19f390 con 0x7f0cdc101fe0 2024-04-25T13:12:30.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.653+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cdc108270 con 0x7f0cdc101fe0 2024-04-25T13:12:30.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.654+0000 7f0cd8ff9640 1 -- 172.21.15.72:0/2126085839 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0cd0042030 con 0x7f0cdc101fe0 2024-04-25T13:12:30.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.654+0000 7f0cd8ff9640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0cc40a3c40 0x7f0cc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.655+0000 7f0cdb7fe640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0cc40a3c40 0x7f0cc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.655+0000 7f0cd8ff9640 1 -- 172.21.15.72:0/2126085839 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f0cd00f0fb0 con 0x7f0cdc101fe0 2024-04-25T13:12:30.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.655+0000 7f0cdb7fe640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0cc40a3c40 0x7f0cc40a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0058c0 tx=0x7f0ccc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:30.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.658+0000 7f0cd8ff9640 1 -- 172.21.15.72:0/2126085839 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cd00bd820 con 0x7f0cdc101fe0 2024-04-25T13:12:30.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.871+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cdc02d5c0 con 0x7f0cdc101fe0 2024-04-25T13:12:30.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.872+0000 7f0cd8ff9640 1 -- 172.21.15.72:0/2126085839 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11881 (secure 0 0 0) 0x7f0cd00bd120 con 0x7f0cdc101fe0 2024-04-25T13:12:30.872 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:30.872 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:11:04.601639+0000","modified":"2024-04-25T13:11:09.292132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24493,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6839/1444196308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1444196308},{"type":"v1","addr":"172.21.15.164:6839","nonce":1444196308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24478,24493,24509]},"id":1}]} 2024-04-25T13:12:30.873 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 20 2024-04-25T13:12:30.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0cc40a3c40 msgr2=0x7f0cc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0cc40a3c40 0x7f0cc40a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0058c0 tx=0x7f0ccc005850 comp rx=0 tx=0).stop 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 msgr2=0x7f0cdc197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 0x7f0cdc197620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0006010 tx=0x7f0cd00042f0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 shutdown_connections 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f0cc40a3c40 0x7f0cc40a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cdc1029a0 0x7f0cdc19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cdc101fe0 0x7f0cdc197620 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 --2- 172.21.15.72:0/2126085839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cdc100e90 0x7f0cdc1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.874+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 >> 172.21.15.72:0/2126085839 conn(0x7f0cdc0fc400 msgr2=0x7f0cdc105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.875+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 shutdown_connections 2024-04-25T13:12:30.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.875+0000 7f0ce1e4e640 1 -- 172.21.15.72:0/2126085839 wait complete. 2024-04-25T13:12:30.886 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:11:04.601639+0000', 'modified': '2024-04-25T13:11:09.292132+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24493, 'mds_2': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/3356589609', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 3356589609}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 3356589609}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/1113925217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 1113925217}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 1113925217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6837/1335235896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 1335235896}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1335235896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.193:6837/1407296685', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1407296685}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1407296685}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.193:6839/2585421276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 2585421276}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 2585421276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6839/1444196308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 1444196308}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 1444196308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24478, 24493, 24509]} 2024-04-25T13:12:30.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T13:12:30.887 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.985+0000 7f149fb20640 1 -- 172.21.15.72:0/2292955638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1498101fe0 msgr2=0x7f1498102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.985+0000 7f149fb20640 1 --2- 172.21.15.72:0/2292955638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1498101fe0 0x7f1498102460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1494009900 tx=0x7f149402a410 comp rx=0 tx=0).stop 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.985+0000 7f149fb20640 1 -- 172.21.15.72:0/2292955638 shutdown_connections 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.985+0000 7f149fb20640 1 --2- 172.21.15.72:0/2292955638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14981029a0 0x7f14981092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.985+0000 7f149fb20640 1 --2- 172.21.15.72:0/2292955638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1498101fe0 0x7f1498102460 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.985+0000 7f149fb20640 1 --2- 172.21.15.72:0/2292955638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1498100e90 0x7f14981012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.985+0000 7f149fb20640 1 -- 172.21.15.72:0/2292955638 >> 172.21.15.72:0/2292955638 conn(0x7f14980fc400 msgr2=0x7f14980fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.985+0000 7f149fb20640 1 -- 172.21.15.72:0/2292955638 shutdown_connections 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.985+0000 7f149fb20640 1 -- 172.21.15.72:0/2292955638 wait complete. 2024-04-25T13:12:30.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149fb20640 1 Processor -- start 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149fb20640 1 -- start start 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149fb20640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1498100e90 0x7f1498069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149fb20640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1498101fe0 0x7f149819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149fb20640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14981029a0 0x7f149819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149fb20640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f149810ca60 con 0x7f14981029a0 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149fb20640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f149810c8e0 con 0x7f1498101fe0 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149fb20640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f149810cbe0 con 0x7f1498100e90 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149e0bd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14981029a0 0x7f149819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149e0bd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14981029a0 0x7f149819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34634/0 (socket says 172.21.15.72:34634) 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149e0bd640 1 -- 172.21.15.72:0/683435448 learned_addr learned my addr 172.21.15.72:0/683435448 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.986+0000 7f149d8bc640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1498100e90 0x7f1498069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f149e0bd640 1 -- 172.21.15.72:0/683435448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1498100e90 msgr2=0x7f1498069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f149e0bd640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1498100e90 0x7f1498069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f149e0bd640 1 -- 172.21.15.72:0/683435448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1498101fe0 msgr2=0x7f149819cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:30.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f149e0bd640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1498101fe0 0x7f149819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f149e0bd640 1 -- 172.21.15.72:0/683435448 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149819fee0 con 0x7f14981029a0 2024-04-25T13:12:30.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f149d8bc640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1498100e90 0x7f1498069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:30.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f149e0bd640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14981029a0 0x7f149819f5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f149000eaa0 tx=0x7f149000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:30.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f148effd640 1 -- 172.21.15.72:0/683435448 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149000cdb0 con 0x7f14981029a0 2024-04-25T13:12:30.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f148effd640 1 -- 172.21.15.72:0/683435448 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1490005020 con 0x7f14981029a0 2024-04-25T13:12:30.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f149fb20640 1 -- 172.21.15.72:0/683435448 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14981a0170 con 0x7f14981029a0 2024-04-25T13:12:30.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f148effd640 1 -- 172.21.15.72:0/683435448 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14900116d0 con 0x7f14981029a0 2024-04-25T13:12:30.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.987+0000 7f149fb20640 1 -- 172.21.15.72:0/683435448 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14981a0740 con 0x7f14981029a0 2024-04-25T13:12:30.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.989+0000 7f148effd640 1 -- 172.21.15.72:0/683435448 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f14900051d0 con 0x7f14981029a0 2024-04-25T13:12:30.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.989+0000 7f14a283b740 1 Processor -- start 2024-04-25T13:12:30.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f14a283b740 1 -- start start 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f14a283b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e0769fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f14a283b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076ad50 0x5598e076b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f14a283b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076b710 0x5598e0771fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f14a283b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598e06cec50 con 0x5598e0769b80 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f14a283b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5598e066b830 con 0x5598e076b710 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f14a283b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5598e06cac10 con 0x5598e076ad50 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e0769fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f148effd640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14840a3c40 0x7f14840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e0769fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34646/0 (socket says 172.21.15.72:34646) 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 -- 172.21.15.72:0/2364169834 learned_addr learned my addr 172.21.15.72:0/2364169834 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149e0bd640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14840a3c40 0x7f14840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d8bc640 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076ad50 0x5598e076b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076b710 0x5598e0771fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 -- 172.21.15.72:0/2364169834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076ad50 msgr2=0x5598e076b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076ad50 0x5598e076b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f148effd640 1 -- 172.21.15.72:0/683435448 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f14900d0810 con 0x7f14981029a0 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 -- 172.21.15.72:0/2364169834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076b710 msgr2=0x5598e0771fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076b710 0x5598e0771fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 -- 172.21.15.72:0/2364169834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e03e12d0 con 0x5598e0769b80 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d8bc640 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076ad50 0x5598e076b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149d0bb640 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e0769fa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1494031500 tx=0x7f14940058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e2d10fda8d5ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.990+0000 7f149e0bd640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14840a3c40 0x7f14840a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f149009c910 tx=0x7f14900c7a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f1483fff640 1 -- 172.21.15.72:0/2364169834 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494044070 con 0x5598e0769b80 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f1483fff640 1 -- 172.21.15.72:0/2364169834 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14940318a0 con 0x5598e0769b80 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f1483fff640 1 -- 172.21.15.72:0/2364169834 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494035040 con 0x5598e0769b80 2024-04-25T13:12:30.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f14a283b740 1 -- 172.21.15.72:0/2364169834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 msgr2=0x5598e0769fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f14a283b740 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e0769fa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1494031500 tx=0x7f14940058d0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f14a283b740 1 -- 172.21.15.72:0/2364169834 shutdown_connections 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f14a283b740 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076b710 0x5598e0771fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f14a283b740 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076ad50 0x5598e076b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f14a283b740 1 --2- 172.21.15.72:0/2364169834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e0769fa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f14a283b740 1 -- 172.21.15.72:0/2364169834 >> 172.21.15.72:0/2364169834 conn(0x5598e0765070 msgr2=0x5598e0765470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f14a283b740 1 -- 172.21.15.72:0/2364169834 shutdown_connections 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.991+0000 7f14a283b740 1 -- 172.21.15.72:0/2364169834 wait complete. 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f14a283b740 1 -- start start 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f14a283b740 1 Processor -- start 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f14a283b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e0769b80 0x5598e0774010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f14a283b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e076ad50 0x5598e0774550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f14a283b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076b710 0x5598e077b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f14a283b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598e07726c0 con 0x5598e076ad50 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f14a283b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5598e0669470 con 0x5598e076b710 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f14a283b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5598e0772840 con 0x5598e0769b80 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f14a283b740 10 client.0 _ll_register_callbacks cb 0x5598e068ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d8bc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e076ad50 0x5598e0774550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d8bc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e076ad50 0x5598e0774550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34656/0 (socket says 172.21.15.72:34656) 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d8bc640 1 -- 172.21.15.72:0/1213057065 learned_addr learned my addr 172.21.15.72:0/1213057065 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d0bb640 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e0769b80 0x5598e0774010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d8bc640 1 -- 172.21.15.72:0/1213057065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e0769b80 msgr2=0x5598e0774010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d8bc640 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e0769b80 0x5598e0774010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d8bc640 1 -- 172.21.15.72:0/1213057065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076b710 msgr2=0x5598e077b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d8bc640 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076b710 0x5598e077b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d8bc640 1 -- 172.21.15.72:0/1213057065 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e04ab6b0 con 0x5598e076ad50 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.992+0000 7f149d0bb640 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e0769b80 0x5598e0774010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:31.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f149d8bc640 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e076ad50 0x5598e0774550 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f148800bfa0 tx=0x7f148800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4048c07fcf8d816 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14817fa640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14817fa640 1 -- 172.21.15.72:0/1213057065 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488018020 con 0x5598e076ad50 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14817fa640 1 -- 172.21.15.72:0/1213057065 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1488014660 con 0x5598e076ad50 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14a283b740 1 -- 172.21.15.72:0/1213057065 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14940318a0 con 0x5598e076ad50 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14817fa640 1 -- 172.21.15.72:0/1213057065 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1488023050 con 0x5598e076ad50 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14a283b740 10 client.15054 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14a283b740 1 -- 172.21.15.72:0/1213057065 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5598e077e9d0 con 0x5598e076ad50 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14817fa640 1 -- 172.21.15.72:0/1213057065 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14940318a0 con 0x5598e076ad50 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14817fa640 1 -- 172.21.15.72:0/1213057065 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f14880139c0 con 0x5598e076ad50 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14a283b740 10 client.15054 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14a283b740 10 client.15054 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14a283b740 20 client.15054 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:31.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14a283b740 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5598e0781080 0x5598e07824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14a283b740 4 client.15054 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.993+0000 7f14a283b740 1 -- 172.21.15.72:0/1213057065 --> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598e06dc670 con 0x5598e0781080 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.994+0000 7f149d0bb640 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5598e0781080 0x5598e07824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.994+0000 7f149d0bb640 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5598e0781080 0x5598e07824f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f546fd933a4b94de server_cookie=ada920d23eb1b206 in_seq=0 out_seq=0 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.994+0000 7f14817fa640 10 client.15054 ms_handle_connect on v2:172.21.15.193:6836/1407296685 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14817fa640 1 -- 172.21.15.72:0/1213057065 <== mds.2 v2:172.21.15.193:6836/1407296685 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598e06dc670 con 0x5598e0781080 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14817fa640 10 client.15054 handle_command_reply: tid=0 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 client.15054 shutdown 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 -- 172.21.15.72:0/1213057065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e076ad50 msgr2=0x5598e0774550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e076ad50 0x5598e0774550 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f148800bfa0 tx=0x7f148800f670 comp rx=0 tx=0).stop 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 -- 172.21.15.72:0/1213057065 shutdown_connections 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5598e0781080 0x5598e07824f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076b710 0x5598e077b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14817fa640 0 client.15054 ms_handle_reset on v2:172.21.15.193:6836/1407296685 2024-04-25T13:12:31.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e076ad50 0x5598e0774550 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 --2- 172.21.15.72:0/1213057065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e0769b80 0x5598e0774010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 -- 172.21.15.72:0/1213057065 >> 172.21.15.72:0/1213057065 conn(0x5598e0765070 msgr2=0x5598e07641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 -- 172.21.15.72:0/1213057065 shutdown_connections 2024-04-25T13:12:31.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 1 -- 172.21.15.72:0/1213057065 wait complete. 2024-04-25T13:12:31.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:30.995+0000 7f14a283b740 20 client.15054 trim_cache size 0 max 16384 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f14a283b740 1 Processor -- start 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f14a283b740 1 -- start start 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f14a283b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e07600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f14a283b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076ad50 0x5598e077b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f14a283b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076b710 0x5598e0767d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f14a283b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598e0780830 con 0x5598e0769b80 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f14a283b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5598e0780f00 con 0x5598e076ad50 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f14a283b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5598e07815d0 con 0x5598e076b710 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f149d0bb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e07600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f149d0bb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e07600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34658/0 (socket says 172.21.15.72:34658) 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f149d0bb640 1 -- 172.21.15.72:0/2423709789 learned_addr learned my addr 172.21.15.72:0/2423709789 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f149d0bb640 1 -- 172.21.15.72:0/2423709789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076b710 msgr2=0x5598e0767d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f149d0bb640 1 --2- 172.21.15.72:0/2423709789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076b710 0x5598e0767d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f149d0bb640 1 -- 172.21.15.72:0/2423709789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076ad50 msgr2=0x5598e077b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f149d0bb640 1 --2- 172.21.15.72:0/2423709789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076ad50 0x5598e077b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f149d0bb640 1 -- 172.21.15.72:0/2423709789 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e076a7d0 con 0x5598e0769b80 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.013+0000 7f149d0bb640 1 --2- 172.21.15.72:0/2423709789 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e07600b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1494009230 tx=0x7f1494009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b995537711fa31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14837fe640 1 -- 172.21.15.72:0/2423709789 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494044040 con 0x5598e0769b80 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14837fe640 1 -- 172.21.15.72:0/2423709789 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1494053070 con 0x5598e0769b80 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14837fe640 1 -- 172.21.15.72:0/2423709789 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494035040 con 0x5598e0769b80 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 -- 172.21.15.72:0/2423709789 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 msgr2=0x5598e07600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 --2- 172.21.15.72:0/2423709789 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e07600b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1494009230 tx=0x7f1494009260 comp rx=0 tx=0).stop 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 -- 172.21.15.72:0/2423709789 shutdown_connections 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 --2- 172.21.15.72:0/2423709789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076b710 0x5598e0767d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 --2- 172.21.15.72:0/2423709789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076ad50 0x5598e077b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 --2- 172.21.15.72:0/2423709789 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e07600b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 -- 172.21.15.72:0/2423709789 >> 172.21.15.72:0/2423709789 conn(0x5598e0765070 msgr2=0x5598e076dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 -- 172.21.15.72:0/2423709789 shutdown_connections 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 -- 172.21.15.72:0/2423709789 wait complete. 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.014+0000 7f14a283b740 1 -- start start 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14a283b740 1 Processor -- start 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14a283b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e076d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14a283b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076ad50 0x5598e076db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14a283b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076b710 0x5598e0779bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14a283b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598e0782370 con 0x5598e0769b80 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14a283b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5598e0781ca0 con 0x5598e076ad50 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14a283b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5598e0782a40 con 0x5598e076b710 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14a283b740 10 client.0 _ll_register_callbacks cb 0x7f148800b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f149d0bb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e076d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f149d0bb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e076d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34668/0 (socket says 172.21.15.72:34668) 2024-04-25T13:12:31.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f149d0bb640 1 -- 172.21.15.72:0/1102497399 learned_addr learned my addr 172.21.15.72:0/1102497399 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f149d0bb640 1 -- 172.21.15.72:0/1102497399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076b710 msgr2=0x5598e0779bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f149d0bb640 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076b710 0x5598e0779bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f149d0bb640 1 -- 172.21.15.72:0/1102497399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076ad50 msgr2=0x5598e076db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f149d0bb640 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076ad50 0x5598e076db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f149d0bb640 1 -- 172.21.15.72:0/1102497399 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e077a230 con 0x5598e0769b80 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f149d0bb640 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e076d5f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1494032e80 tx=0x7f1494031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca495361ff1887b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14817fa640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.015+0000 7f14817fa640 1 -- 172.21.15.72:0/1102497399 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494035070 con 0x5598e0769b80 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14817fa640 1 -- 172.21.15.72:0/1102497399 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1494004040 con 0x5598e0769b80 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14a283b740 1 -- 172.21.15.72:0/1102497399 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1494053070 con 0x5598e0769b80 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14817fa640 1 -- 172.21.15.72:0/1102497399 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1494044040 con 0x5598e0769b80 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14a283b740 10 client.15060 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14a283b740 1 -- 172.21.15.72:0/1102497399 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5598e077b4f0 con 0x5598e0769b80 2024-04-25T13:12:31.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14817fa640 1 -- 172.21.15.72:0/1102497399 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1494053070 con 0x5598e0769b80 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14817fa640 1 -- 172.21.15.72:0/1102497399 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f14940035f0 con 0x5598e0769b80 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14a283b740 10 client.15060 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14a283b740 10 client.15060 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14a283b740 20 client.15060 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14a283b740 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5598e07eaf40 0x5598e07e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14a283b740 4 client.15060 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f14a283b740 1 -- 172.21.15.72:0/1102497399 --> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] -- command(tid 0: {"prefix": "status"}) -- 0x5598e06d64e0 con 0x5598e07eaf40 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.016+0000 7f149d8bc640 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5598e07eaf40 0x5598e07e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14817fa640 10 client.15060 ms_handle_connect on v2:172.21.15.193:6836/1407296685 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f149d8bc640 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5598e07eaf40 0x5598e07e9c50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c30d1621941ee2bf server_cookie=d3d6f3ce3ae88efa in_seq=0 out_seq=0 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14817fa640 1 -- 172.21.15.72:0/1102497399 <== mds.2 v2:172.21.15.193:6836/1407296685 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5598e06d64e0 con 0x5598e07eaf40 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14817fa640 10 client.15060 handle_command_reply: tid=0 2024-04-25T13:12:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 client.15060 shutdown 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 -- 172.21.15.72:0/1102497399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 msgr2=0x5598e076d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e076d5f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1494032e80 tx=0x7f1494031a70 comp rx=0 tx=0).stop 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 -- 172.21.15.72:0/1102497399 shutdown_connections 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598e076b710 0x5598e0779bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5598e07eaf40 0x5598e07e9c50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598e076ad50 0x5598e076db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 --2- 172.21.15.72:0/1102497399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598e0769b80 0x5598e076d5f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14817fa640 0 client.15060 ms_handle_reset on v2:172.21.15.193:6836/1407296685 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 -- 172.21.15.72:0/1102497399 >> 172.21.15.72:0/1102497399 conn(0x5598e0765070 msgr2=0x5598e07600b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 -- 172.21.15.72:0/1102497399 shutdown_connections 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 1 -- 172.21.15.72:0/1102497399 wait complete. 2024-04-25T13:12:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.017+0000 7f14a283b740 20 client.15060 trim_cache size 0 max 16384 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 2, 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24478, 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 82.734009068999995, 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 20, 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 90, 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T13:12:31.024 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 105.79708116499999 2024-04-25T13:12:31.025 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:12:31.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 -- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14840a3c40 msgr2=0x7f14840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14840a3c40 0x7f14840a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f149009c910 tx=0x7f14900c7a90 comp rx=0 tx=0).stop 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 -- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14981029a0 msgr2=0x7f149819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14981029a0 0x7f149819f5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f149000eaa0 tx=0x7f149000ef60 comp rx=0 tx=0).stop 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 -- 172.21.15.72:0/683435448 shutdown_connections 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f14840a3c40 0x7f14840a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14981029a0 0x7f149819f5d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1498101fe0 0x7f149819cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 --2- 172.21.15.72:0/683435448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1498100e90 0x7f1498069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 -- 172.21.15.72:0/683435448 >> 172.21.15.72:0/683435448 conn(0x7f14980fc400 msgr2=0x7f1498105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 -- 172.21.15.72:0/683435448 shutdown_connections 2024-04-25T13:12:31.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.018+0000 7f149fb20640 1 -- 172.21.15.72:0/683435448 wait complete. 2024-04-25T13:12:31.029 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:12:31.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 -- 172.21.15.72:0/2680406352 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 msgr2=0x7ffaec1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 --2- 172.21.15.72:0/2680406352 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 0x7ffaec1092c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009ca0 tx=0x7ffae402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 -- 172.21.15.72:0/2680406352 shutdown_connections 2024-04-25T13:12:31.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 --2- 172.21.15.72:0/2680406352 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 0x7ffaec1092c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 --2- 172.21.15.72:0/2680406352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffaec101fe0 0x7ffaec102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 --2- 172.21.15.72:0/2680406352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaec100e90 0x7ffaec1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 -- 172.21.15.72:0/2680406352 >> 172.21.15.72:0/2680406352 conn(0x7ffaec0fc400 msgr2=0x7ffaec0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 -- 172.21.15.72:0/2680406352 shutdown_connections 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 -- 172.21.15.72:0/2680406352 wait complete. 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 Processor -- start 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 -- start start 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffaec100e90 0x7ffaec069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaec101fe0 0x7ffaec19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 0x7ffaec19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec10c9d0 con 0x7ffaec1029a0 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec10c850 con 0x7ffaec101fe0 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.128+0000 7ffaf3376640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec10cb50 con 0x7ffaec100e90 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1913640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 0x7ffaec19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1913640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 0x7ffaec19f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34676/0 (socket says 172.21.15.72:34676) 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1913640 1 -- 172.21.15.72:0/1466606167 learned_addr learned my addr 172.21.15.72:0/1466606167 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf0911640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaec101fe0 0x7ffaec19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1112640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffaec100e90 0x7ffaec069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1913640 1 -- 172.21.15.72:0/1466606167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffaec100e90 msgr2=0x7ffaec069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1913640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffaec100e90 0x7ffaec069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1913640 1 -- 172.21.15.72:0/1466606167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaec101fe0 msgr2=0x7ffaec19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1913640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaec101fe0 0x7ffaec19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1913640 1 -- 172.21.15.72:0/1466606167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec19fdc0 con 0x7ffaec1029a0 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf1913640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 0x7ffaec19f4b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffae40097b0 tx=0x7ffae4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffae27fc640 1 -- 172.21.15.72:0/1466606167 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4004240 con 0x7ffaec1029a0 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffae27fc640 1 -- 172.21.15.72:0/1466606167 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae40317c0 con 0x7ffaec1029a0 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf3376640 1 -- 172.21.15.72:0/1466606167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec19fff0 con 0x7ffaec1029a0 2024-04-25T13:12:31.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffae27fc640 1 -- 172.21.15.72:0/1466606167 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae402fd80 con 0x7ffaec1029a0 2024-04-25T13:12:31.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.129+0000 7ffaf3376640 1 -- 172.21.15.72:0/1466606167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec1a0500 con 0x7ffaec1029a0 2024-04-25T13:12:31.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.131+0000 7ffae27fc640 1 -- 172.21.15.72:0/1466606167 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffae40043e0 con 0x7ffaec1029a0 2024-04-25T13:12:31.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf616f740 1 Processor -- start 2024-04-25T13:12:31.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf616f740 1 -- start start 2024-04-25T13:12:31.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffae27fc640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffad80a3cc0 0x7ffad80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e7afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7bd50 0x559f71e7c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7c710 0x559f71e82fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf616f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559f71ddfc50 con 0x559f71e7bd50 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf616f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559f71d7c830 con 0x559f71e7ab80 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf616f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f71ddbc10 con 0x559f71e7c710 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf0911640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffad80a3cc0 0x7ffad80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffae27fc640 1 -- 172.21.15.72:0/1466606167 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ffae40f1ea0 con 0x7ffaec1029a0 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf0911640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7bd50 0x559f71e7c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf0911640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7bd50 0x559f71e7c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34680/0 (socket says 172.21.15.72:34680) 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf0911640 1 -- 172.21.15.72:0/2535563339 learned_addr learned my addr 172.21.15.72:0/2535563339 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf1112640 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e7afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.132+0000 7ffaf1913640 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7c710 0x559f71e82fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffaf0911640 1 -- 172.21.15.72:0/2535563339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7c710 msgr2=0x559f71e82fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffaf0911640 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7c710 0x559f71e82fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffaf0911640 1 -- 172.21.15.72:0/2535563339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 msgr2=0x559f71e7afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffaf0911640 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e7afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffaf0911640 1 -- 172.21.15.72:0/2535563339 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f71af22d0 con 0x559f71e7bd50 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffaf1913640 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7c710 0x559f71e82fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffaf0911640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffad80a3cc0 0x7ffad80a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffaec100bb0 tx=0x7ffadc00bae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffaf0911640 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7bd50 0x559f71e7c1d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffadc010c90 tx=0x7ffadc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd0e371e2c771f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffad77fe640 1 -- 172.21.15.72:0/2535563339 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc021070 con 0x559f71e7bd50 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffad77fe640 1 -- 172.21.15.72:0/2535563339 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffadc00b360 con 0x559f71e7bd50 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.133+0000 7ffad77fe640 1 -- 172.21.15.72:0/2535563339 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc005020 con 0x559f71e7bd50 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 -- 172.21.15.72:0/2535563339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7bd50 msgr2=0x559f71e7c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7bd50 0x559f71e7c1d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffadc010c90 tx=0x7ffadc01a000 comp rx=0 tx=0).stop 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 -- 172.21.15.72:0/2535563339 shutdown_connections 2024-04-25T13:12:31.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7c710 0x559f71e82fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7bd50 0x559f71e7c1d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2535563339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e7afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 -- 172.21.15.72:0/2535563339 >> 172.21.15.72:0/2535563339 conn(0x559f71e76070 msgr2=0x559f71e76470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 -- 172.21.15.72:0/2535563339 shutdown_connections 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 -- 172.21.15.72:0/2535563339 wait complete. 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 -- start start 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.134+0000 7ffaf616f740 1 Processor -- start 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e85050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e8c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559f71e836c0 con 0x559f71e7c710 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559f71d7a470 con 0x559f71e7ab80 2024-04-25T13:12:31.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f71e83840 con 0x559f71e7bd50 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 10 client.0 _ll_register_callbacks cb 0x559f71d9ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1913640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e8c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1913640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e8c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34682/0 (socket says 172.21.15.72:34682) 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1913640 1 -- 172.21.15.72:0/2432099866 learned_addr learned my addr 172.21.15.72:0/2432099866 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1112640 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e85050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf0911640 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1913640 1 -- 172.21.15.72:0/2432099866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 msgr2=0x559f71e85590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1913640 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e85590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1913640 1 -- 172.21.15.72:0/2432099866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 msgr2=0x559f71e85050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1913640 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e85050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1913640 1 -- 172.21.15.72:0/2432099866 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f71bbc6b0 con 0x559f71e7c710 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf1913640 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e8c5d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffae4045000 tx=0x7ffae4086fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ae41a437f79a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffad4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffad4ff9640 1 -- 172.21.15.72:0/2432099866 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4003e10 con 0x559f71e7c710 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffad4ff9640 1 -- 172.21.15.72:0/2432099866 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ffae40efa30 con 0x559f71e7c710 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 1 -- 172.21.15.72:0/2432099866 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffadc00b360 con 0x559f71e7c710 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffad4ff9640 1 -- 172.21.15.72:0/2432099866 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ffae40be8e0 con 0x559f71e7c710 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 10 client.15072 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.135+0000 7ffaf616f740 1 -- 172.21.15.72:0/2432099866 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559f71e836c0 con 0x559f71e7c710 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.136+0000 7ffad4ff9640 1 -- 172.21.15.72:0/2432099866 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffadc00b360 con 0x559f71e7c710 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.136+0000 7ffad4ff9640 1 -- 172.21.15.72:0/2432099866 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7ffae40c2120 con 0x559f71e7c710 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.136+0000 7ffaf616f740 10 client.15072 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.136+0000 7ffaf616f740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.136+0000 7ffaf616f740 20 client.15072 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.136+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x559f71e92060 0x559f71e934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.136+0000 7ffaf616f740 4 client.15072 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:12:31.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.136+0000 7ffaf616f740 1 -- 172.21.15.72:0/2432099866 --> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f71ded670 con 0x559f71e92060 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.136+0000 7ffaf1112640 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x559f71e92060 0x559f71e934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.137+0000 7ffad4ff9640 10 client.15072 ms_handle_connect on v2:172.21.15.193:6838/2585421276 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.137+0000 7ffaf1112640 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x559f71e92060 0x559f71e934d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16849dec5f278e9 server_cookie=1e32eda6ca5587e7 in_seq=0 out_seq=0 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.137+0000 7ffad4ff9640 1 -- 172.21.15.72:0/2432099866 <== mds.1 v2:172.21.15.193:6838/2585421276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f71ded670 con 0x559f71e92060 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.137+0000 7ffad4ff9640 10 client.15072 handle_command_reply: tid=0 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.137+0000 7ffaf616f740 1 client.15072 shutdown 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.137+0000 7ffaf616f740 1 -- 172.21.15.72:0/2432099866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 msgr2=0x559f71e8c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.137+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e8c5d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffae4045000 tx=0x7ffae4086fd0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffaf616f740 1 -- 172.21.15.72:0/2432099866 shutdown_connections 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x559f71e92060 0x559f71e934d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e8c5d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffad4ff9640 0 client.15072 ms_handle_reset on v2:172.21.15.193:6838/2585421276 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e85590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffaf616f740 1 --2- 172.21.15.72:0/2432099866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e85050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffaf616f740 1 -- 172.21.15.72:0/2432099866 >> 172.21.15.72:0/2432099866 conn(0x559f71e76070 msgr2=0x559f71e751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffaf616f740 1 -- 172.21.15.72:0/2432099866 shutdown_connections 2024-04-25T13:12:31.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffaf616f740 1 -- 172.21.15.72:0/2432099866 wait complete. 2024-04-25T13:12:31.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.138+0000 7ffaf616f740 20 client.15072 trim_cache size 0 max 16384 2024-04-25T13:12:31.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.155+0000 7ffaf616f740 1 Processor -- start 2024-04-25T13:12:31.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.155+0000 7ffaf616f740 1 -- start start 2024-04-25T13:12:31.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e84670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e8c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf616f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559f71e91810 con 0x559f71e7c710 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf616f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559f71e91ee0 con 0x559f71e7ab80 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf616f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f71e925b0 con 0x559f71e7bd50 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf1913640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf1913640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34696/0 (socket says 172.21.15.72:34696) 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf1913640 1 -- 172.21.15.72:0/3275658405 learned_addr learned my addr 172.21.15.72:0/3275658405 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf0911640 1 --2- 172.21.15.72:0/3275658405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e8c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf1913640 1 -- 172.21.15.72:0/3275658405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 msgr2=0x559f71e8c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf1913640 1 --2- 172.21.15.72:0/3275658405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e8c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf1913640 1 -- 172.21.15.72:0/3275658405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 msgr2=0x559f71e84670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf1913640 1 --2- 172.21.15.72:0/3275658405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e84670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf1913640 1 -- 172.21.15.72:0/3275658405 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f71e85ef0 con 0x559f71e7c710 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffaf1913640 1 --2- 172.21.15.72:0/3275658405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e710b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffae40be170 tx=0x7ffae40bc6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be2abcfad7188cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffad6ffd640 1 -- 172.21.15.72:0/3275658405 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae409e070 con 0x559f71e7c710 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffad6ffd640 1 -- 172.21.15.72:0/3275658405 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4036040 con 0x559f71e7c710 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.156+0000 7ffad6ffd640 1 -- 172.21.15.72:0/3275658405 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae40c6e50 con 0x559f71e7c710 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 -- 172.21.15.72:0/3275658405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 msgr2=0x559f71e710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3275658405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e710b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffae40be170 tx=0x7ffae40bc6f0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 -- 172.21.15.72:0/3275658405 shutdown_connections 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3275658405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7c710 0x559f71e710b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3275658405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e8c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3275658405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7ab80 0x559f71e84670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 -- 172.21.15.72:0/3275658405 >> 172.21.15.72:0/3275658405 conn(0x559f71e76070 msgr2=0x559f71e7ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 -- 172.21.15.72:0/3275658405 shutdown_connections 2024-04-25T13:12:31.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 -- 172.21.15.72:0/3275658405 wait complete. 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 -- start start 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.157+0000 7ffaf616f740 1 Processor -- start 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7ab80 0x559f71e88570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e8aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7c710 0x559f71e8b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559f71e93350 con 0x559f71e7ab80 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559f71e92c80 con 0x559f71e7c710 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f71e93a20 con 0x559f71e7bd50 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 10 client.0 _ll_register_callbacks cb 0x7ffae40ba480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1112640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7ab80 0x559f71e88570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1112640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7ab80 0x559f71e88570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34708/0 (socket says 172.21.15.72:34708) 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1112640 1 -- 172.21.15.72:0/3100171335 learned_addr learned my addr 172.21.15.72:0/3100171335 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf0911640 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e8aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1913640 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7c710 0x559f71e8b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1112640 1 -- 172.21.15.72:0/3100171335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 msgr2=0x559f71e8aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1112640 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e8aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1112640 1 -- 172.21.15.72:0/3100171335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7c710 msgr2=0x559f71e8b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1112640 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7c710 0x559f71e8b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1112640 1 -- 172.21.15.72:0/3100171335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f71e8b640 con 0x559f71e7ab80 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf0911640 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e8aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1913640 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7c710 0x559f71e8b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf1112640 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7ab80 0x559f71e88570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x559f71e7bc50 tx=0x7ffae80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ea7fa7c91df3e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffad4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffad4ff9640 1 -- 172.21.15.72:0/3100171335 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8021030 con 0x559f71e7ab80 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffad4ff9640 1 -- 172.21.15.72:0/3100171335 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ffae801d420 con 0x559f71e7ab80 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 1 -- 172.21.15.72:0/3100171335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae4036040 con 0x559f71e7ab80 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffad4ff9640 1 -- 172.21.15.72:0/3100171335 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ffae8003f80 con 0x559f71e7ab80 2024-04-25T13:12:31.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 10 client.15078 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffaf616f740 1 -- 172.21.15.72:0/3100171335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559f71ef8010 con 0x559f71e7ab80 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.158+0000 7ffad4ff9640 1 -- 172.21.15.72:0/3100171335 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4036040 con 0x559f71e7ab80 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffad4ff9640 1 -- 172.21.15.72:0/3100171335 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7ffae8011440 con 0x559f71e7ab80 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffaf616f740 10 client.15078 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffaf616f740 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffaf616f740 20 client.15078 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x559f71efca50 0x559f71efb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffaf616f740 4 client.15078 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffaf616f740 1 -- 172.21.15.72:0/3100171335 --> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] -- command(tid 0: {"prefix": "status"}) -- 0x559f71de74e0 con 0x559f71efca50 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffaf0911640 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x559f71efca50 0x559f71efb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffad4ff9640 10 client.15078 ms_handle_connect on v2:172.21.15.193:6838/2585421276 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.159+0000 7ffaf0911640 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x559f71efca50 0x559f71efb760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a30a6b5375f17511 server_cookie=9a6604cbd5f430e3 in_seq=0 out_seq=0 2024-04-25T13:12:31.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffad4ff9640 1 -- 172.21.15.72:0/3100171335 <== mds.1 v2:172.21.15.193:6838/2585421276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559f71de74e0 con 0x559f71efca50 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffad4ff9640 10 client.15078 handle_command_reply: tid=0 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 client.15078 shutdown 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 -- 172.21.15.72:0/3100171335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7ab80 msgr2=0x559f71e88570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7ab80 0x559f71e88570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x559f71e7bc50 tx=0x7ffae80035f0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 -- 172.21.15.72:0/3100171335 shutdown_connections 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x559f71efca50 0x559f71efb760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f71e7c710 0x559f71e8b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f71e7bd50 0x559f71e8aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffad4ff9640 0 client.15078 ms_handle_reset on v2:172.21.15.193:6838/2585421276 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 --2- 172.21.15.72:0/3100171335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559f71e7ab80 0x559f71e88570 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 -- 172.21.15.72:0/3100171335 >> 172.21.15.72:0/3100171335 conn(0x559f71e76070 msgr2=0x559f71e90ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 -- 172.21.15.72:0/3100171335 shutdown_connections 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 1 -- 172.21.15.72:0/3100171335 wait complete. 2024-04-25T13:12:31.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf616f740 20 client.15078 trim_cache size 0 max 16384 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 1, 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24493, 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 83.904807438000006, 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 20, 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 90, 2024-04-25T13:12:31.168 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T13:12:31.169 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 100.753858208 2024-04-25T13:12:31.169 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:12:31.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf3376640 1 -- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffad80a3cc0 msgr2=0x7ffad80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf3376640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffad80a3cc0 0x7ffad80a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffaec100bb0 tx=0x7ffadc00bae0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf3376640 1 -- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 msgr2=0x7ffaec19f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.160+0000 7ffaf3376640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 0x7ffaec19f4b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffae40097b0 tx=0x7ffae4003c20 comp rx=0 tx=0).stop 2024-04-25T13:12:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.161+0000 7ffaf3376640 1 -- 172.21.15.72:0/1466606167 shutdown_connections 2024-04-25T13:12:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.161+0000 7ffaf3376640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ffad80a3cc0 0x7ffad80a6180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.161+0000 7ffaf3376640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffaec1029a0 0x7ffaec19f4b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.161+0000 7ffaf3376640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaec101fe0 0x7ffaec19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.161+0000 7ffaf3376640 1 --2- 172.21.15.72:0/1466606167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffaec100e90 0x7ffaec069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.161+0000 7ffaf3376640 1 -- 172.21.15.72:0/1466606167 >> 172.21.15.72:0/1466606167 conn(0x7ffaec0fc400 msgr2=0x7ffaec1052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.161+0000 7ffaf3376640 1 -- 172.21.15.72:0/1466606167 shutdown_connections 2024-04-25T13:12:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.161+0000 7ffaf3376640 1 -- 172.21.15.72:0/1466606167 wait complete. 2024-04-25T13:12:31.171 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:12:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 -- 172.21.15.72:0/729532917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094102a90 msgr2=0x7f4094104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 --2- 172.21.15.72:0/729532917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094102a90 0x7f4094104e80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f408c009ca0 tx=0x7f408c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 -- 172.21.15.72:0/729532917 shutdown_connections 2024-04-25T13:12:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 --2- 172.21.15.72:0/729532917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4094107cf0 0x7f409410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 --2- 172.21.15.72:0/729532917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40941053c0 0x7f40941077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 --2- 172.21.15.72:0/729532917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094102a90 0x7f4094104e80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 -- 172.21.15.72:0/729532917 >> 172.21.15.72:0/729532917 conn(0x7f40940fc440 msgr2=0x7f40940fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 -- 172.21.15.72:0/729532917 shutdown_connections 2024-04-25T13:12:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 -- 172.21.15.72:0/729532917 wait complete. 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 Processor -- start 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 -- start start 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4094102a90 0x7f4094198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.272+0000 7f4099995640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40941053c0 0x7f40941987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4099995640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094107cf0 0x7f409419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4099995640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f409410db50 con 0x7f4094107cf0 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4099995640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f409410d9d0 con 0x7f40941053c0 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4099995640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f409410dcd0 con 0x7f4094102a90 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40937fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094107cf0 0x7f409419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40937fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094107cf0 0x7f409419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34722/0 (socket says 172.21.15.72:34722) 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40937fe640 1 -- 172.21.15.72:0/2101953239 learned_addr learned my addr 172.21.15.72:0/2101953239 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40927fc640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40941053c0 0x7f40941987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4092ffd640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4094102a90 0x7f4094198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40937fe640 1 -- 172.21.15.72:0/2101953239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4094102a90 msgr2=0x7f4094198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40937fe640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4094102a90 0x7f4094198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40937fe640 1 -- 172.21.15.72:0/2101953239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40941053c0 msgr2=0x7f40941987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40937fe640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40941053c0 0x7f40941987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40937fe640 1 -- 172.21.15.72:0/2101953239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409419fea0 con 0x7f4094107cf0 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40927fc640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40941053c0 0x7f40941987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f40937fe640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094107cf0 0x7f409419f830 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4084009fd0 tx=0x7f408400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4098993640 1 -- 172.21.15.72:0/2101953239 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084011830 con 0x7f4094107cf0 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4099995640 1 -- 172.21.15.72:0/2101953239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40941a0190 con 0x7f4094107cf0 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4098993640 1 -- 172.21.15.72:0/2101953239 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4084016070 con 0x7f4094107cf0 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4098993640 1 -- 172.21.15.72:0/2101953239 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084010de0 con 0x7f4094107cf0 2024-04-25T13:12:31.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.273+0000 7f4099995640 1 -- 172.21.15.72:0/2101953239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40941a07b0 con 0x7f4094107cf0 2024-04-25T13:12:31.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.275+0000 7f4098993640 1 -- 172.21.15.72:0/2101953239 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f40840119d0 con 0x7f4094107cf0 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f409c785740 1 Processor -- start 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f409c785740 1 -- start start 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f409c785740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c66f90 0x55ef32c673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f409c785740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68160 0x55ef32c685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f409c785740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 0x55ef32c6f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f409c785740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ef32bc7c50 con 0x55ef32c68160 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f409c785740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ef32b64830 con 0x55ef32c66f90 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f409c785740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ef32bc3c10 con 0x55ef32c68b20 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4098993640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f40800a3c40 0x7f40800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4092ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68160 0x55ef32c685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4092ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68160 0x55ef32c685e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34724/0 (socket says 172.21.15.72:34724) 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4092ffd640 1 -- 172.21.15.72:0/2130317782 learned_addr learned my addr 172.21.15.72:0/2130317782 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f40937fe640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f40800a3c40 0x7f40800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4092ffd640 1 -- 172.21.15.72:0/2130317782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 msgr2=0x55ef32c6f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f40927fc640 1 --2- 172.21.15.72:0/2130317782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 0x55ef32c6f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4098993640 1 -- 172.21.15.72:0/2101953239 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f40840cfbe0 con 0x7f4094107cf0 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4092ffd640 1 --2- 172.21.15.72:0/2130317782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 0x55ef32c6f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4092ffd640 1 -- 172.21.15.72:0/2130317782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c66f90 msgr2=0x55ef32c673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4092ffd640 1 --2- 172.21.15.72:0/2130317782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c66f90 0x55ef32c673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4092ffd640 1 -- 172.21.15.72:0/2130317782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef328da2d0 con 0x55ef32c68160 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f40927fc640 1 --2- 172.21.15.72:0/2130317782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 0x55ef32c6f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f4092ffd640 1 --2- 172.21.15.72:0/2130317782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68160 0x55ef32c685e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f408c035000 tx=0x7f408c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf772de7d4f3cea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.276+0000 7f40937fe640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f40800a3c40 0x7f40800a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f408409bb60 tx=0x7f4084066950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f4089ffb640 1 -- 172.21.15.72:0/2130317782 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c003970 con 0x55ef32c68160 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f4089ffb640 1 -- 172.21.15.72:0/2130317782 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408c036040 con 0x55ef32c68160 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f4089ffb640 1 -- 172.21.15.72:0/2130317782 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c02f660 con 0x55ef32c68160 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f409c785740 1 -- 172.21.15.72:0/2130317782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68160 msgr2=0x55ef32c685e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f409c785740 1 --2- 172.21.15.72:0/2130317782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68160 0x55ef32c685e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f408c035000 tx=0x7f408c0099b0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f409c785740 1 -- 172.21.15.72:0/2130317782 shutdown_connections 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f409c785740 1 --2- 172.21.15.72:0/2130317782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 0x55ef32c6f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f409c785740 1 --2- 172.21.15.72:0/2130317782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68160 0x55ef32c685e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f409c785740 1 --2- 172.21.15.72:0/2130317782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c66f90 0x55ef32c673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f409c785740 1 -- 172.21.15.72:0/2130317782 >> 172.21.15.72:0/2130317782 conn(0x55ef32c5e070 msgr2=0x55ef32c5e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f409c785740 1 -- 172.21.15.72:0/2130317782 shutdown_connections 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.277+0000 7f409c785740 1 -- 172.21.15.72:0/2130317782 wait complete. 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f409c785740 1 -- start start 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f409c785740 1 Processor -- start 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f409c785740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c66f90 0x55ef32c71360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f409c785740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 0x55ef32c718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f409c785740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68b20 0x55ef32c788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f409c785740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ef32c6fad0 con 0x55ef32c68b20 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f409c785740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ef32b62470 con 0x55ef32c66f90 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f409c785740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ef32c6fc50 con 0x55ef32c68160 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f409c785740 10 client.0 _ll_register_callbacks cb 0x55ef32b87f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f40927fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68b20 0x55ef32c788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f40927fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68b20 0x55ef32c788e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34728/0 (socket says 172.21.15.72:34728) 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f40927fc640 1 -- 172.21.15.72:0/1051977797 learned_addr learned my addr 172.21.15.72:0/1051977797 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f40927fc640 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c66f90 0x55ef32c71360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f4092ffd640 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 0x55ef32c718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f40927fc640 1 -- 172.21.15.72:0/1051977797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 msgr2=0x55ef32c718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f40927fc640 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 0x55ef32c718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f40927fc640 1 -- 172.21.15.72:0/1051977797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c66f90 msgr2=0x55ef32c71360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f40927fc640 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c66f90 0x55ef32c71360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.278+0000 7f40927fc640 1 -- 172.21.15.72:0/1051977797 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef329a46b0 con 0x55ef32c68b20 2024-04-25T13:12:31.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f40927fc640 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68b20 0x55ef32c788e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f407c012e10 tx=0x7f407c00c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55a74a495d89bda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f4072ffd640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f4072ffd640 1 -- 172.21.15.72:0/1051977797 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c00cc60 con 0x55ef32c68b20 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f4072ffd640 1 -- 172.21.15.72:0/1051977797 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f407c00ce00 con 0x55ef32c68b20 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f409c785740 1 -- 172.21.15.72:0/1051977797 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408c036040 con 0x55ef32c68b20 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f4072ffd640 1 -- 172.21.15.72:0/1051977797 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f407c025050 con 0x55ef32c68b20 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f409c785740 10 client.15090 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f409c785740 1 -- 172.21.15.72:0/1051977797 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ef32c7bc90 con 0x55ef32c68b20 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f4072ffd640 1 -- 172.21.15.72:0/1051977797 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408c036040 con 0x55ef32c68b20 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f4072ffd640 1 -- 172.21.15.72:0/1051977797 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f407c003c60 con 0x55ef32c68b20 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f409c785740 10 client.15090 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f409c785740 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f409c785740 20 client.15090 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f409c785740 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x55ef32c7e2a0 0x55ef32c7f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f409c785740 4 client.15090 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.279+0000 7f409c785740 1 -- 172.21.15.72:0/1051977797 --> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef32bd5670 con 0x55ef32c7e2a0 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.280+0000 7f4092ffd640 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x55ef32c7e2a0 0x55ef32c7f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.280+0000 7f4092ffd640 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x55ef32c7e2a0 0x55ef32c7f710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c86b45c0d3b8518 server_cookie=21becd20e41f92ce in_seq=0 out_seq=0 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.280+0000 7f4072ffd640 10 client.15090 ms_handle_connect on v2:172.21.15.164:6838/1444196308 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f4072ffd640 1 -- 172.21.15.72:0/1051977797 <== mds.0 v2:172.21.15.164:6838/1444196308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef32bd5670 con 0x55ef32c7e2a0 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f4072ffd640 10 client.15090 handle_command_reply: tid=0 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 client.15090 shutdown 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 -- 172.21.15.72:0/1051977797 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68b20 msgr2=0x55ef32c788e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68b20 0x55ef32c788e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f407c012e10 tx=0x7f407c00c690 comp rx=0 tx=0).stop 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 -- 172.21.15.72:0/1051977797 shutdown_connections 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x55ef32c7e2a0 0x55ef32c7f710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c68b20 0x55ef32c788e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 0x55ef32c718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 --2- 172.21.15.72:0/1051977797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c66f90 0x55ef32c71360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f4072ffd640 0 client.15090 ms_handle_reset on v2:172.21.15.164:6838/1444196308 2024-04-25T13:12:31.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 -- 172.21.15.72:0/1051977797 >> 172.21.15.72:0/1051977797 conn(0x55ef32c5e070 msgr2=0x55ef32c5d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 -- 172.21.15.72:0/1051977797 shutdown_connections 2024-04-25T13:12:31.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 1 -- 172.21.15.72:0/1051977797 wait complete. 2024-04-25T13:12:31.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.281+0000 7f409c785740 20 client.15090 trim_cache size 0 max 16384 2024-04-25T13:12:31.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f409c785740 1 Processor -- start 2024-04-25T13:12:31.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f409c785740 1 -- start start 2024-04-25T13:12:31.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f409c785740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c68540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f409c785740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c68160 0x55ef32c78800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f409c785740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 0x55ef32c70f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f409c785740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ef32c7da50 con 0x55ef32c66f90 2024-04-25T13:12:31.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f409c785740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ef32c7e120 con 0x55ef32c68160 2024-04-25T13:12:31.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f409c785740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ef32c7e7f0 con 0x55ef32c68b20 2024-04-25T13:12:31.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40927fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c68540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40927fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c68540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34744/0 (socket says 172.21.15.72:34744) 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40927fc640 1 -- 172.21.15.72:0/2098929671 learned_addr learned my addr 172.21.15.72:0/2098929671 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40927fc640 1 -- 172.21.15.72:0/2098929671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 msgr2=0x55ef32c70f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40927fc640 1 --2- 172.21.15.72:0/2098929671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 0x55ef32c70f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40927fc640 1 -- 172.21.15.72:0/2098929671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c68160 msgr2=0x55ef32c78800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40927fc640 1 --2- 172.21.15.72:0/2098929671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c68160 0x55ef32c78800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40927fc640 1 -- 172.21.15.72:0/2098929671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef32c6eec0 con 0x55ef32c66f90 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40927fc640 1 --2- 172.21.15.72:0/2098929671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c68540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f407c012de0 tx=0x7f407c010600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442a4504d58a8d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40897fa640 1 -- 172.21.15.72:0/2098929671 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c010700 con 0x55ef32c66f90 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40897fa640 1 -- 172.21.15.72:0/2098929671 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f407c02e070 con 0x55ef32c66f90 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.299+0000 7f40897fa640 1 -- 172.21.15.72:0/2098929671 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c00ce80 con 0x55ef32c66f90 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 -- 172.21.15.72:0/2098929671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 msgr2=0x55ef32c68540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 --2- 172.21.15.72:0/2098929671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c68540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f407c012de0 tx=0x7f407c010600 comp rx=0 tx=0).stop 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 -- 172.21.15.72:0/2098929671 shutdown_connections 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 --2- 172.21.15.72:0/2098929671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68b20 0x55ef32c70f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 --2- 172.21.15.72:0/2098929671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c68160 0x55ef32c78800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 --2- 172.21.15.72:0/2098929671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c68540 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 -- 172.21.15.72:0/2098929671 >> 172.21.15.72:0/2098929671 conn(0x55ef32c5e070 msgr2=0x55ef32c6bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 -- 172.21.15.72:0/2098929671 shutdown_connections 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 -- 172.21.15.72:0/2098929671 wait complete. 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.300+0000 7f409c785740 1 -- start start 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f409c785740 1 Processor -- start 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f409c785740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c74180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f409c785740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 0x55ef32c746c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f409c785740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c68b20 0x55ef32c77690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f409c785740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ef32c7f590 con 0x55ef32c66f90 2024-04-25T13:12:31.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f409c785740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ef32c7eec0 con 0x55ef32c68b20 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f409c785740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ef32c7fc60 con 0x55ef32c68160 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f409c785740 10 client.0 _ll_register_callbacks cb 0x7f407c00c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f40927fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c74180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f40927fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c74180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34758/0 (socket says 172.21.15.72:34758) 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f40927fc640 1 -- 172.21.15.72:0/1738300928 learned_addr learned my addr 172.21.15.72:0/1738300928 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f4092ffd640 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 0x55ef32c746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f40927fc640 1 -- 172.21.15.72:0/1738300928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 msgr2=0x55ef32c746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f40927fc640 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 0x55ef32c746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f40927fc640 1 -- 172.21.15.72:0/1738300928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c68b20 msgr2=0x55ef32c77690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f40927fc640 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c68b20 0x55ef32c77690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f40927fc640 1 -- 172.21.15.72:0/1738300928 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef32c77c80 con 0x55ef32c66f90 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f40927fc640 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c74180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f407c012e10 tx=0x7f407c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e15ba4670d2443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:31.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f4072ffd640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f4072ffd640 1 -- 172.21.15.72:0/1738300928 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c010a00 con 0x55ef32c66f90 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f4072ffd640 1 -- 172.21.15.72:0/1738300928 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f407c010ba0 con 0x55ef32c66f90 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.301+0000 7f409c785740 1 -- 172.21.15.72:0/1738300928 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f407c02e070 con 0x55ef32c66f90 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f4072ffd640 1 -- 172.21.15.72:0/1738300928 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f407c010da0 con 0x55ef32c66f90 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f409c785740 10 client.15096 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f409c785740 1 -- 172.21.15.72:0/1738300928 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ef32ce44d0 con 0x55ef32c66f90 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f4072ffd640 1 -- 172.21.15.72:0/1738300928 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f407c02e070 con 0x55ef32c66f90 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f4072ffd640 1 -- 172.21.15.72:0/1738300928 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f407c004680 con 0x55ef32c66f90 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f409c785740 10 client.15096 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f409c785740 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f409c785740 20 client.15096 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f409c785740 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x55ef32ce8ea0 0x55ef32ce7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f409c785740 4 client.15096 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:12:31.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f409c785740 1 -- 172.21.15.72:0/1738300928 --> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] -- command(tid 0: {"prefix": "status"}) -- 0x55ef32bcf4e0 con 0x55ef32ce8ea0 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.302+0000 7f4092ffd640 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x55ef32ce8ea0 0x55ef32ce7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f4072ffd640 10 client.15096 ms_handle_connect on v2:172.21.15.164:6838/1444196308 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f4092ffd640 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x55ef32ce8ea0 0x55ef32ce7bb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f38929a1983877 server_cookie=59006996f0282c8a in_seq=0 out_seq=0 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f4072ffd640 1 -- 172.21.15.72:0/1738300928 <== mds.0 v2:172.21.15.164:6838/1444196308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7f408c054e50 con 0x55ef32ce8ea0 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f4072ffd640 10 client.15096 handle_command_reply: tid=0 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 client.15096 shutdown 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 -- 172.21.15.72:0/1738300928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 msgr2=0x55ef32c74180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c74180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f407c012e10 tx=0x7f407c00cb50 comp rx=0 tx=0).stop 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 -- 172.21.15.72:0/1738300928 shutdown_connections 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x55ef32ce8ea0 0x55ef32ce7bb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef32c68b20 0x55ef32c77690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f4072ffd640 0 client.15096 ms_handle_reset on v2:172.21.15.164:6838/1444196308 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef32c68160 0x55ef32c746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 --2- 172.21.15.72:0/1738300928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef32c66f90 0x55ef32c74180 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 -- 172.21.15.72:0/1738300928 >> 172.21.15.72:0/1738300928 conn(0x55ef32c5e070 msgr2=0x55ef32c70f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 -- 172.21.15.72:0/1738300928 shutdown_connections 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 1 -- 172.21.15.72:0/1738300928 wait complete. 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f409c785740 20 client.15096 trim_cache size 0 max 16384 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24509, 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-25T13:12:31.310 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-25T13:12:31.311 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-25T13:12:31.311 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 86.037100237000004, 2024-04-25T13:12:31.311 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 20, 2024-04-25T13:12:31.311 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 90, 2024-04-25T13:12:31.311 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T13:12:31.311 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 92.613760576999994 2024-04-25T13:12:31.311 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f4099995640 1 -- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f40800a3c40 msgr2=0x7f40800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f4099995640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f40800a3c40 0x7f40800a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f408409bb60 tx=0x7f4084066950 comp rx=0 tx=0).stop 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f4099995640 1 -- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094107cf0 msgr2=0x7f409419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.303+0000 7f4099995640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094107cf0 0x7f409419f830 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4084009fd0 tx=0x7f408400b8e0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.304+0000 7f4099995640 1 -- 172.21.15.72:0/2101953239 shutdown_connections 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.304+0000 7f4099995640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f40800a3c40 0x7f40800a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.304+0000 7f4099995640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4094107cf0 0x7f409419f830 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.304+0000 7f4099995640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40941053c0 0x7f40941987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.304+0000 7f4099995640 1 --2- 172.21.15.72:0/2101953239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4094102a90 0x7f4094198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.304+0000 7f4099995640 1 -- 172.21.15.72:0/2101953239 >> 172.21.15.72:0/2101953239 conn(0x7f40940fc440 msgr2=0x7f40941033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:31.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.304+0000 7f4099995640 1 -- 172.21.15.72:0/2101953239 shutdown_connections 2024-04-25T13:12:31.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:31.304+0000 7f4099995640 1 -- 172.21.15.72:0/2101953239 wait complete. 2024-04-25T13:12:31.315 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T13:12:31.315 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:12:31.324 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:12:31.324 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T13:12:31.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:31 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/4160508952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:31 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2085106592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:31 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2126085839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:31 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/4160508952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:31 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2085106592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:31 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2126085839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:31 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/4160508952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:31 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2085106592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:31 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2126085839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.560 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:32.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:32 smithi193 ceph-mon[32743]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:12:32.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:32 smithi072 ceph-mon[29199]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:12:32.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:32 smithi164 ceph-mon[32418]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.164+0000 7f71188b5700 1 -- 172.21.15.72:0/1847074047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114103560 msgr2=0x7f7114109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.164+0000 7f71188b5700 1 --2- 172.21.15.72:0/1847074047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114103560 0x7f7114109e10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f710c00b0d0 tx=0x7f710c00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.164+0000 7f71188b5700 1 -- 172.21.15.72:0/1847074047 shutdown_connections 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.164+0000 7f71188b5700 1 --2- 172.21.15.72:0/1847074047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114103560 0x7f7114109e10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 --2- 172.21.15.72:0/1847074047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71141019c0 0x7f7114101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 --2- 172.21.15.72:0/1847074047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7114102bc0 0x7f7114103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 -- 172.21.15.72:0/1847074047 >> 172.21.15.72:0/1847074047 conn(0x7f71140fcf60 msgr2=0x7f71140ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 -- 172.21.15.72:0/1847074047 shutdown_connections 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 -- 172.21.15.72:0/1847074047 wait complete. 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 Processor -- start 2024-04-25T13:12:33.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 -- start start 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71141019c0 0x7f711406bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114102bc0 0x7f711419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7114103560 0x7f71141a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f711410d550 con 0x7f7114102bc0 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f711410d3d0 con 0x7f71141019c0 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.165+0000 7f71188b5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f711410d6d0 con 0x7f7114103560 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7111d9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114102bc0 0x7f711419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7111d9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114102bc0 0x7f711419da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34786/0 (socket says 172.21.15.72:34786) 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7111d9b700 1 -- 172.21.15.72:0/4011256818 learned_addr learned my addr 172.21.15.72:0/4011256818 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f711259c700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71141019c0 0x7f711406bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7112d9d700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7114103560 0x7f71141a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:33.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7111d9b700 1 -- 172.21.15.72:0/4011256818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7114103560 msgr2=0x7f71141a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7111d9b700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7114103560 0x7f71141a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7111d9b700 1 -- 172.21.15.72:0/4011256818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71141019c0 msgr2=0x7f711406bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7111d9b700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71141019c0 0x7f711406bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7111d9b700 1 -- 172.21.15.72:0/4011256818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71141a0960 con 0x7f7114102bc0 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f7111d9b700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114102bc0 0x7f711419da50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f710800b530 tx=0x7f710800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f71037fe700 1 -- 172.21.15.72:0/4011256818 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108015020 con 0x7f7114102bc0 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71141a0c50 con 0x7f7114102bc0 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f71037fe700 1 -- 172.21.15.72:0/4011256818 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7108011ce0 con 0x7f7114102bc0 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f71037fe700 1 -- 172.21.15.72:0/4011256818 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108010e40 con 0x7f7114102bc0 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.166+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71141a1140 con 0x7f7114102bc0 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.167+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7114069730 con 0x7f7114102bc0 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.168+0000 7f71037fe700 1 -- 172.21.15.72:0/4011256818 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7108018020 con 0x7f7114102bc0 2024-04-25T13:12:33.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.169+0000 7f71037fe700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f70fc0a3cd0 0x7f70fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:33.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.169+0000 7f711259c700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f70fc0a3cd0 0x7f70fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:33.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.169+0000 7f71037fe700 1 -- 172.21.15.72:0/4011256818 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f71080cfd20 con 0x7f7114102bc0 2024-04-25T13:12:33.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.169+0000 7f711259c700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f70fc0a3cd0 0x7f70fc0a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7114102a20 tx=0x7f7104009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:33.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.173+0000 7f71037fe700 1 -- 172.21.15.72:0/4011256818 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f710801f020 con 0x7f7114102bc0 2024-04-25T13:12:33.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.397+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7114062e80 con 0x7f70fc0a3cd0 2024-04-25T13:12:33.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.398+0000 7f71037fe700 1 -- 172.21.15.72:0/4011256818 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7114062e80 con 0x7f70fc0a3cd0 2024-04-25T13:12:33.397 INFO:teuthology.orchestra.run.smithi072.stdout:Backend: cephadm 2024-04-25T13:12:33.397 INFO:teuthology.orchestra.run.smithi072.stdout:Available: Yes 2024-04-25T13:12:33.397 INFO:teuthology.orchestra.run.smithi072.stdout:Paused: No 2024-04-25T13:12:33.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f70fc0a3cd0 msgr2=0x7f70fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:33.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f70fc0a3cd0 0x7f70fc0a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7114102a20 tx=0x7f7104009190 comp rx=0 tx=0).stop 2024-04-25T13:12:33.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114102bc0 msgr2=0x7f711419da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:33.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114102bc0 0x7f711419da50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f710800b530 tx=0x7f710800b840 comp rx=0 tx=0).stop 2024-04-25T13:12:33.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 shutdown_connections 2024-04-25T13:12:33.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7114103560 0x7f71141a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71141019c0 0x7f711406bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f70fc0a3cd0 0x7f70fc0a6190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 --2- 172.21.15.72:0/4011256818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7114102bc0 0x7f711419da50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:33.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 >> 172.21.15.72:0/4011256818 conn(0x7f71140fcf60 msgr2=0x7f7114105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:33.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 shutdown_connections 2024-04-25T13:12:33.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:33.400+0000 7f71188b5700 1 -- 172.21.15.72:0/4011256818 wait complete. 2024-04-25T13:12:34.105 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T13:12:34.341 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:34.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:34 smithi072 ceph-mon[29199]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:34.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:34 smithi193 ceph-mon[32743]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:34.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:34 smithi164 ceph-mon[32418]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:35.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:35 smithi193 ceph-mon[32743]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:35.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:35 smithi072 ceph-mon[29199]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:35.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:35 smithi164 ceph-mon[32418]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:35.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.986+0000 7fbc71d4b700 1 -- 172.21.15.72:0/401778080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c1046e0 msgr2=0x7fbc6c10af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:35.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.986+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/401778080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c1046e0 0x7fbc6c10af90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b0a0 tx=0x7fbc6400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:12:35.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.986+0000 7fbc71d4b700 1 -- 172.21.15.72:0/401778080 shutdown_connections 2024-04-25T13:12:35.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.986+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/401778080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c1046e0 0x7fbc6c10af90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:35.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.986+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/401778080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c100d60 0x7fbc6c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:35.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.986+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/401778080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c103d40 0x7fbc6c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:35.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.986+0000 7fbc71d4b700 1 -- 172.21.15.72:0/401778080 >> 172.21.15.72:0/401778080 conn(0x7fbc6c0fc6a0 msgr2=0x7fbc6c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:35.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.986+0000 7fbc71d4b700 1 -- 172.21.15.72:0/401778080 shutdown_connections 2024-04-25T13:12:35.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.986+0000 7fbc71d4b700 1 -- 172.21.15.72:0/401778080 wait complete. 2024-04-25T13:12:35.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc71d4b700 1 Processor -- start 2024-04-25T13:12:35.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc71d4b700 1 -- start start 2024-04-25T13:12:35.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc71d4b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c100d60 0x7fbc6c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:35.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc71d4b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c103d40 0x7fbc6c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:35.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc71d4b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c1046e0 0x7fbc6c1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:35.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc71d4b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c10e530 con 0x7fbc6c103d40 2024-04-25T13:12:35.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc71d4b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c10e3b0 con 0x7fbc6c100d60 2024-04-25T13:12:35.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc71d4b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c10e6b0 con 0x7fbc6c1046e0 2024-04-25T13:12:35.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc6affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c103d40 0x7fbc6c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:35.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc6affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c103d40 0x7fbc6c1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56188/0 (socket says 172.21.15.72:56188) 2024-04-25T13:12:35.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc6affd700 1 -- 172.21.15.72:0/548085091 learned_addr learned my addr 172.21.15.72:0/548085091 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:35.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc6b7fe700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c100d60 0x7fbc6c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:35.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.987+0000 7fbc6bfff700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c1046e0 0x7fbc6c1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:35.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc6affd700 1 -- 172.21.15.72:0/548085091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c1046e0 msgr2=0x7fbc6c1a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:35.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc6affd700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c1046e0 0x7fbc6c1a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc6affd700 1 -- 172.21.15.72:0/548085091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c100d60 msgr2=0x7fbc6c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc6affd700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c100d60 0x7fbc6c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc6affd700 1 -- 172.21.15.72:0/548085091 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c1a07b0 con 0x7fbc6c103d40 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc6bfff700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c1046e0 0x7fbc6c1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc6affd700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c103d40 0x7fbc6c1990e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000e9d0 tx=0x7fbc6000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc68ff9700 1 -- 172.21.15.72:0/548085091 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6000cbd0 con 0x7fbc6c103d40 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c1a0aa0 con 0x7fbc6c103d40 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc68ff9700 1 -- 172.21.15.72:0/548085091 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc6000cd70 con 0x7fbc6c103d40 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc68ff9700 1 -- 172.21.15.72:0/548085091 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60018680 con 0x7fbc6c103d40 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.988+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c1a0fe0 con 0x7fbc6c103d40 2024-04-25T13:12:35.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.989+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6c069730 con 0x7fbc6c103d40 2024-04-25T13:12:35.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.990+0000 7fbc68ff9700 1 -- 172.21.15.72:0/548085091 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbc60010430 con 0x7fbc6c103d40 2024-04-25T13:12:35.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.991+0000 7fbc68ff9700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fbc540a3cd0 0x7fbc540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:35.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.991+0000 7fbc6b7fe700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fbc540a3cd0 0x7fbc540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:35.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.991+0000 7fbc68ff9700 1 -- 172.21.15.72:0/548085091 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fbc600d07e0 con 0x7fbc6c103d40 2024-04-25T13:12:35.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.991+0000 7fbc6b7fe700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fbc540a3cd0 0x7fbc540a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c06b010 tx=0x7fbc5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:35.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:35.995+0000 7fbc68ff9700 1 -- 172.21.15.72:0/548085091 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc6009d050 con 0x7fbc6c103d40 2024-04-25T13:12:36.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.211+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbc6c062e80 con 0x7fbc540a3cd0 2024-04-25T13:12:36.225 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.225+0000 7fbc68ff9700 1 -- 172.21.15.72:0/548085091 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fbc6c062e80 con 0x7fbc540a3cd0 2024-04-25T13:12:36.225 INFO:teuthology.orchestra.run.smithi072.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T13:12:36.225 INFO:teuthology.orchestra.run.smithi072.stdout:mds.a smithi072 running (100s) 17s ago 100s 16.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1d514d7aff4a 2024-04-25T13:12:36.225 INFO:teuthology.orchestra.run.smithi072.stdout:mds.b smithi164 running (97s) 31s ago 97s 26.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 32fe3eaa6717 2024-04-25T13:12:36.225 INFO:teuthology.orchestra.run.smithi072.stdout:mds.c smithi193 running (111s) 31s ago 110s 16.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d985942e3e7c 2024-04-25T13:12:36.225 INFO:teuthology.orchestra.run.smithi072.stdout:mds.d smithi072 running (119s) 17s ago 118s 14.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d4657a749527 2024-04-25T13:12:36.225 INFO:teuthology.orchestra.run.smithi072.stdout:mds.e smithi164 running (116s) 31s ago 116s 15.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 857c80292701 2024-04-25T13:12:36.225 INFO:teuthology.orchestra.run.smithi072.stdout:mds.f smithi193 running (113s) 31s ago 113s 16.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9aab990b038b 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mds.g smithi072 running (103s) 17s ago 102s 15.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5f4ecb322965 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mds.h smithi164 running (108s) 31s ago 108s 18.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9d9ba379a90a 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mds.i smithi193 running (106s) 31s ago 105s 18.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ae363537a1ac 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mds.j smithi072 running (2m) 17s ago 2m 16.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a009131c2d33 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mds.k smithi164 running (2m) 31s ago 2m 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4ac470d933d8 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mds.l smithi193 running (2m) 31s ago 2m 16.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 26563088b66f 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mgr.x smithi072 *:9283,8765 running (9m) 17s ago 9m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fafea5b7b7d7 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mgr.y smithi164 *:8443,8765 running (7m) 31s ago 7m 453M - 19.0.0-3156-gd69f13d8 da4b004c09e8 69b024243b70 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mgr.z smithi193 *:8443,8765 running (7m) 31s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d1e176b2ff2e 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mon.a smithi072 running (9m) 17s ago 9m 52.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 f54ab9bf98cb 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mon.b smithi164 running (7m) 31s ago 7m 42.3M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 a54523d0c0ee 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:mon.c smithi193 running (7m) 31s ago 7m 42.1M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 6e1c06de1eb5 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:osd.0 smithi072 running (6m) 17s ago 6m 58.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3be2dc428051 2024-04-25T13:12:36.226 INFO:teuthology.orchestra.run.smithi072.stdout:osd.1 smithi164 running (6m) 31s ago 6m 60.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 5153eb3fcb9b 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.2 smithi193 running (6m) 31s ago 6m 58.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f66947b8701a 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.3 smithi072 running (5m) 17s ago 5m 46.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 fc1c2fc3b01d 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.4 smithi164 running (5m) 31s ago 5m 45.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 a96f148cd4f0 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.5 smithi193 running (4m) 31s ago 4m 54.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 8b33258ef408 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.6 smithi072 running (4m) 17s ago 4m 55.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 eae38ecbc546 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.7 smithi164 running (4m) 31s ago 4m 45.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 aa6d0b0f2128 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.8 smithi193 running (3m) 31s ago 3m 54.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ef88b21be693 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.9 smithi072 running (3m) 17s ago 3m 46.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 9754ac952a63 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.10 smithi164 running (2m) 31s ago 2m 58.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 631450c60eb5 2024-04-25T13:12:36.227 INFO:teuthology.orchestra.run.smithi072.stdout:osd.11 smithi193 running (2m) 31s ago 2m 42.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 0318ccf2853e 2024-04-25T13:12:36.228 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fbc540a3cd0 msgr2=0x7fbc540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:36.228 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fbc540a3cd0 0x7fbc540a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c06b010 tx=0x7fbc5c009260 comp rx=0 tx=0).stop 2024-04-25T13:12:36.228 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c103d40 msgr2=0x7fbc6c1990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:36.228 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c103d40 0x7fbc6c1990e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000e9d0 tx=0x7fbc6000ece0 comp rx=0 tx=0).stop 2024-04-25T13:12:36.228 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 shutdown_connections 2024-04-25T13:12:36.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c1046e0 0x7fbc6c1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:36.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c100d60 0x7fbc6c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:36.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fbc540a3cd0 0x7fbc540a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:36.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 --2- 172.21.15.72:0/548085091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc6c103d40 0x7fbc6c1990e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:36.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 >> 172.21.15.72:0/548085091 conn(0x7fbc6c0fc6a0 msgr2=0x7fbc6c1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:36.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 shutdown_connections 2024-04-25T13:12:36.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:36.228+0000 7fbc71d4b700 1 -- 172.21.15.72:0/548085091 wait complete. 2024-04-25T13:12:36.299 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:36 smithi072 ceph-mon[29199]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:36.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:36 smithi164 ceph-mon[32418]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:36.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:36 smithi193 ceph-mon[32743]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:36.930 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T13:12:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:37.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:37 smithi072 ceph-mon[29199]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:37.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:37 smithi164 ceph-mon[32418]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:37.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:37 smithi193 ceph-mon[32743]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:38.378 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:38 smithi072 ceph-mon[29199]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:38.379 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:38 smithi072 ceph-mon[29199]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:12:38.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:38 smithi164 ceph-mon[32418]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:38.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:38 smithi164 ceph-mon[32418]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:12:38.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:38 smithi193 ceph-mon[32743]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:38.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:38 smithi193 ceph-mon[32743]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:12:38.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 -- 172.21.15.72:0/4204536581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 msgr2=0x7ff904109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:38.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 --2- 172.21.15.72:0/4204536581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 0x7ff904109df0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00b0d0 tx=0x7ff8fc00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:12:38.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 -- 172.21.15.72:0/4204536581 shutdown_connections 2024-04-25T13:12:38.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 --2- 172.21.15.72:0/4204536581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 0x7ff904109df0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 --2- 172.21.15.72:0/4204536581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9041019a0 0x7ff904101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 --2- 172.21.15.72:0/4204536581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff904102ba0 0x7ff904103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 -- 172.21.15.72:0/4204536581 >> 172.21.15.72:0/4204536581 conn(0x7ff9040fcf60 msgr2=0x7ff9040ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 -- 172.21.15.72:0/4204536581 shutdown_connections 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 -- 172.21.15.72:0/4204536581 wait complete. 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 Processor -- start 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.953+0000 7ff90c609700 1 -- start start 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90c609700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9041019a0 0x7ff904105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90c609700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff904102ba0 0x7ff904103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90c609700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 0x7ff9041044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90c609700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff90410d440 con 0x7ff904103540 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90c609700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff90410d2c0 con 0x7ff9041019a0 2024-04-25T13:12:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90c609700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff90410d5c0 con 0x7ff904102ba0 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90aba6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 0x7ff9041044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90aba6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 0x7ff9041044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56198/0 (socket says 172.21.15.72:56198) 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90aba6700 1 -- 172.21.15.72:0/3030647466 learned_addr learned my addr 172.21.15.72:0/3030647466 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90a3a5700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9041019a0 0x7ff904105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff909ba4700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff904102ba0 0x7ff904103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90aba6700 1 -- 172.21.15.72:0/3030647466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff904102ba0 msgr2=0x7ff904103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90aba6700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff904102ba0 0x7ff904103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90aba6700 1 -- 172.21.15.72:0/3030647466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9041019a0 msgr2=0x7ff904105930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90aba6700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9041019a0 0x7ff904105930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90aba6700 1 -- 172.21.15.72:0/3030647466 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff904104d70 con 0x7ff904103540 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90aba6700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 0x7ff9041044c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009fd0 tx=0x7ff8fc004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff8fb7fe700 1 -- 172.21.15.72:0/3030647466 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc01f030 con 0x7ff904103540 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9041a2c40 con 0x7ff904103540 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.954+0000 7ff8fb7fe700 1 -- 172.21.15.72:0/3030647466 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8fc007a80 con 0x7ff904103540 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.955+0000 7ff8fb7fe700 1 -- 172.21.15.72:0/3030647466 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc004810 con 0x7ff904103540 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.955+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9041a3090 con 0x7ff904103540 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.955+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff904069730 con 0x7ff904103540 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.957+0000 7ff8fb7fe700 1 -- 172.21.15.72:0/3030647466 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff8fc003a00 con 0x7ff904103540 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.957+0000 7ff8fb7fe700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff8f00a3c40 0x7ff8f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.957+0000 7ff90a3a5700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff8f00a3c40 0x7ff8f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.957+0000 7ff8fb7fe700 1 -- 172.21.15.72:0/3030647466 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff8fc00f020 con 0x7ff904103540 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.958+0000 7ff90a3a5700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff8f00a3c40 0x7ff8f00a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff900005950 tx=0x7ff90000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:38.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:38.961+0000 7ff8fb7fe700 1 -- 172.21.15.72:0/3030647466 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8fc029020 con 0x7ff904103540 2024-04-25T13:12:39.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.168+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff9040ff0e0 con 0x7ff8f00a3c40 2024-04-25T13:12:39.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.174+0000 7ff8fb7fe700 1 -- 172.21.15.72:0/3030647466 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff9040ff0e0 con 0x7ff8f00a3c40 2024-04-25T13:12:39.174 INFO:teuthology.orchestra.run.smithi072.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T13:12:39.174 INFO:teuthology.orchestra.run.smithi072.stdout:mds.all 12/12 34s ago 2m smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-04-25T13:12:39.174 INFO:teuthology.orchestra.run.smithi072.stdout:mgr 3/3 34s ago 7m smithi072=x;smithi164=y;smithi193=z;count:3 2024-04-25T13:12:39.174 INFO:teuthology.orchestra.run.smithi072.stdout:mon 3/3 34s ago 7m smithi072:172.21.15.72=a;smithi164:172.21.15.164=b;smithi193:172.21.15.193=c;count:3 2024-04-25T13:12:39.174 INFO:teuthology.orchestra.run.smithi072.stdout:osd 12 34s ago - 2024-04-25T13:12:39.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff8f00a3c40 msgr2=0x7ff8f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:39.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff8f00a3c40 0x7ff8f00a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff900005950 tx=0x7ff90000a300 comp rx=0 tx=0).stop 2024-04-25T13:12:39.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 msgr2=0x7ff9041044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:39.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 0x7ff9041044c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009fd0 tx=0x7ff8fc004e40 comp rx=0 tx=0).stop 2024-04-25T13:12:39.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 shutdown_connections 2024-04-25T13:12:39.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff8f00a3c40 0x7ff8f00a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:39.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff904103540 0x7ff9041044c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:39.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9041019a0 0x7ff904105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:39.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 --2- 172.21.15.72:0/3030647466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff904102ba0 0x7ff904103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:39.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 >> 172.21.15.72:0/3030647466 conn(0x7ff9040fcf60 msgr2=0x7ff9040fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:39.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 shutdown_connections 2024-04-25T13:12:39.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:39.176+0000 7ff90c609700 1 -- 172.21.15.72:0/3030647466 wait complete. 2024-04-25T13:12:39.790 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T13:12:40.031 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:40.404 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:40 smithi072 ceph-mon[29199]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:40.404 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:40 smithi072 ceph-mon[29199]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:40.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:40 smithi164 ceph-mon[32418]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:40.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:40 smithi164 ceph-mon[32418]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:40 smithi193 ceph-mon[32743]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:40 smithi193 ceph-mon[32743]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:41.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.761+0000 7f590f767700 1 -- 172.21.15.72:0/1313719236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f590810a980 msgr2=0x7f590810ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.761+0000 7f590f767700 1 --2- 172.21.15.72:0/1313719236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f590810a980 0x7f590810ce10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f590000b0a0 tx=0x7f590000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.761+0000 7f590f767700 1 -- 172.21.15.72:0/1313719236 shutdown_connections 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.761+0000 7f590f767700 1 --2- 172.21.15.72:0/1313719236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f590810a980 0x7f590810ce10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.761+0000 7f590f767700 1 --2- 172.21.15.72:0/1313719236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5908076a10 0x7f5908074e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.761+0000 7f590f767700 1 --2- 172.21.15.72:0/1313719236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5908075440 0x7f59080758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.761+0000 7f590f767700 1 -- 172.21.15.72:0/1313719236 >> 172.21.15.72:0/1313719236 conn(0x7f59080fe040 msgr2=0x7f5908100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.761+0000 7f590f767700 1 -- 172.21.15.72:0/1313719236 shutdown_connections 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.761+0000 7f590f767700 1 -- 172.21.15.72:0/1313719236 wait complete. 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590f767700 1 Processor -- start 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590f767700 1 -- start start 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590f767700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5908075440 0x7f590819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590f767700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5908076a10 0x7f590819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590f767700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f590810a980 0x7f59081a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:41.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590f767700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f59081106f0 con 0x7f5908076a10 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590f767700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5908110570 con 0x7f5908075440 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590f767700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5908110870 con 0x7f590810a980 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590d503700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5908075440 0x7f590819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590d503700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5908075440 0x7f590819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:41216/0 (socket says 172.21.15.72:41216) 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590d503700 1 -- 172.21.15.72:0/2271953865 learned_addr learned my addr 172.21.15.72:0/2271953865 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590cd02700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5908076a10 0x7f590819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590dd04700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f590810a980 0x7f59081a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590cd02700 1 -- 172.21.15.72:0/2271953865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f590810a980 msgr2=0x7f59081a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590cd02700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f590810a980 0x7f59081a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590cd02700 1 -- 172.21.15.72:0/2271953865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5908075440 msgr2=0x7f590819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590cd02700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5908075440 0x7f590819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590cd02700 1 -- 172.21.15.72:0/2271953865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59081a2980 con 0x7f5908076a10 2024-04-25T13:12:41.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590dd04700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f590810a980 0x7f59081a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:41.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.762+0000 7f590cd02700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5908076a10 0x7f590819b270 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f590400c970 tx=0x7f590400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:41.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.763+0000 7f58fe7fc700 1 -- 172.21.15.72:0/2271953865 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590400ce60 con 0x7f5908076a10 2024-04-25T13:12:41.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.763+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59081a2c70 con 0x7f5908076a10 2024-04-25T13:12:41.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.763+0000 7f58fe7fc700 1 -- 172.21.15.72:0/2271953865 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5904004830 con 0x7f5908076a10 2024-04-25T13:12:41.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.763+0000 7f58fe7fc700 1 -- 172.21.15.72:0/2271953865 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904005420 con 0x7f5908076a10 2024-04-25T13:12:41.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.763+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59081a31b0 con 0x7f5908076a10 2024-04-25T13:12:41.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.764+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5908069730 con 0x7f5908076a10 2024-04-25T13:12:41.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.765+0000 7f58fe7fc700 1 -- 172.21.15.72:0/2271953865 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5904004c70 con 0x7f5908076a10 2024-04-25T13:12:41.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.765+0000 7f58fe7fc700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58f40a3c40 0x7f58f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:41.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.765+0000 7f590d503700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58f40a3c40 0x7f58f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:41.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.766+0000 7f58fe7fc700 1 -- 172.21.15.72:0/2271953865 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f59040d0400 con 0x7f5908076a10 2024-04-25T13:12:41.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.766+0000 7f590d503700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58f40a3c40 0x7f58f40a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f590819c210 tx=0x7f58f800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:41.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.769+0000 7f58fe7fc700 1 -- 172.21.15.72:0/2271953865 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5904014030 con 0x7f5908076a10 2024-04-25T13:12:41.977 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.977+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5908062e80 con 0x7f58f40a3c40 2024-04-25T13:12:41.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.979+0000 7f58fe7fc700 1 -- 172.21.15.72:0/2271953865 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5908062e80 con 0x7f58f40a3c40 2024-04-25T13:12:41.979 INFO:teuthology.orchestra.run.smithi072.stdout:HOST ADDR LABELS STATUS 2024-04-25T13:12:41.979 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072 172.21.15.72 2024-04-25T13:12:41.979 INFO:teuthology.orchestra.run.smithi072.stdout:smithi164 172.21.15.164 2024-04-25T13:12:41.979 INFO:teuthology.orchestra.run.smithi072.stdout:smithi193 172.21.15.193 2024-04-25T13:12:41.979 INFO:teuthology.orchestra.run.smithi072.stdout:3 hosts in cluster 2024-04-25T13:12:41.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58f40a3c40 msgr2=0x7f58f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:41.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58f40a3c40 0x7f58f40a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f590819c210 tx=0x7f58f800b500 comp rx=0 tx=0).stop 2024-04-25T13:12:41.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5908076a10 msgr2=0x7f590819b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:41.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5908076a10 0x7f590819b270 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f590400c970 tx=0x7f590400cc80 comp rx=0 tx=0).stop 2024-04-25T13:12:41.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 shutdown_connections 2024-04-25T13:12:41.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f590810a980 0x7f59081a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5908075440 0x7f590819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f58f40a3c40 0x7f58f40a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 --2- 172.21.15.72:0/2271953865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5908076a10 0x7f590819b270 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:41.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 >> 172.21.15.72:0/2271953865 conn(0x7f59080fe040 msgr2=0x7f59081068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:41.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 shutdown_connections 2024-04-25T13:12:41.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:41.981+0000 7f590f767700 1 -- 172.21.15.72:0/2271953865 wait complete. 2024-04-25T13:12:42.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:42 smithi072 ceph-mon[29199]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T13:12:42.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:42 smithi164 ceph-mon[32418]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T13:12:42.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:42 smithi193 ceph-mon[32743]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T13:12:42.785 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T13:12:43.026 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:43.399 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:43 smithi072 ceph-mon[29199]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:43.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:43 smithi164 ceph-mon[32418]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:43.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:43 smithi193 ceph-mon[32743]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:44.465 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:44 smithi072 ceph-mon[29199]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:12:44.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:44 smithi164 ceph-mon[32418]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:12:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:44 smithi193 ceph-mon[32743]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:12:44.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.797+0000 7ffa0dada700 1 -- 172.21.15.72:0/2673962861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 msgr2=0x7ffa08104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:44.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.797+0000 7ffa0dada700 1 --2- 172.21.15.72:0/2673962861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 0x7ffa08104100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0099c0 tx=0x7ff9fc009cd0 comp rx=0 tx=0).stop 2024-04-25T13:12:44.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.797+0000 7ffa0dada700 1 -- 172.21.15.72:0/2673962861 shutdown_connections 2024-04-25T13:12:44.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.797+0000 7ffa0dada700 1 --2- 172.21.15.72:0/2673962861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa08104640 0x7ffa0810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:44.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.797+0000 7ffa0dada700 1 --2- 172.21.15.72:0/2673962861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa08102aa0 0x7ffa08102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.797+0000 7ffa0dada700 1 --2- 172.21.15.72:0/2673962861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 0x7ffa08104100 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.797+0000 7ffa0dada700 1 -- 172.21.15.72:0/2673962861 >> 172.21.15.72:0/2673962861 conn(0x7ffa080fe040 msgr2=0x7ffa08100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.797+0000 7ffa0dada700 1 -- 172.21.15.72:0/2673962861 shutdown_connections 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.797+0000 7ffa0dada700 1 -- 172.21.15.72:0/2673962861 wait complete. 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa0dada700 1 Processor -- start 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa0dada700 1 -- start start 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa0dada700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa08102aa0 0x7ffa080767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa0dada700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 0x7ffa08074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa0dada700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa08104640 0x7ffa08075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa0dada700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810e540 con 0x7ffa08103ca0 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa0dada700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810e3c0 con 0x7ffa08102aa0 2024-04-25T13:12:44.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa0dada700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810e6c0 con 0x7ffa08104640 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa06ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 0x7ffa08074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa06ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 0x7ffa08074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56238/0 (socket says 172.21.15.72:56238) 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.798+0000 7ffa06ffd700 1 -- 172.21.15.72:0/3462274508 learned_addr learned my addr 172.21.15.72:0/3462274508 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa077fe700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa08102aa0 0x7ffa080767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa07fff700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa08104640 0x7ffa08075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa06ffd700 1 -- 172.21.15.72:0/3462274508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa08104640 msgr2=0x7ffa08075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa06ffd700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa08104640 0x7ffa08075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa06ffd700 1 -- 172.21.15.72:0/3462274508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa08102aa0 msgr2=0x7ffa080767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa06ffd700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa08102aa0 0x7ffa080767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa06ffd700 1 -- 172.21.15.72:0/3462274508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa08075c20 con 0x7ffa08103ca0 2024-04-25T13:12:44.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa07fff700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa08104640 0x7ffa08075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa06ffd700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 0x7ffa08074e30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc003790 tx=0x7ff9fc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa04ff9700 1 -- 172.21.15.72:0/3462274508 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc027070 con 0x7ffa08103ca0 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa04ff9700 1 -- 172.21.15.72:0/3462274508 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc022470 con 0x7ffa08103ca0 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa04ff9700 1 -- 172.21.15.72:0/3462274508 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00f6e0 con 0x7ffa08103ca0 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa081a2c80 con 0x7ffa08103ca0 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.799+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa081a3130 con 0x7ffa08103ca0 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.800+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa08069730 con 0x7ffa08103ca0 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.801+0000 7ffa04ff9700 1 -- 172.21.15.72:0/3462274508 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff9fc02a070 con 0x7ffa08103ca0 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.802+0000 7ffa04ff9700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9f00a3c40 0x7ff9f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:44.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.802+0000 7ffa077fe700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9f00a3c40 0x7ff9f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.802+0000 7ffa04ff9700 1 -- 172.21.15.72:0/3462274508 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff9fc0d1210 con 0x7ffa08103ca0 2024-04-25T13:12:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.802+0000 7ffa077fe700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9f00a3c40 0x7ff9f00a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa08103b00 tx=0x7ff9f800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:44.806+0000 7ffa04ff9700 1 -- 172.21.15.72:0/3462274508 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9fc013070 con 0x7ffa08103ca0 2024-04-25T13:12:45.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.012+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ffa08062e80 con 0x7ff9f00a3c40 2024-04-25T13:12:45.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.014+0000 7ffa04ff9700 1 -- 172.21.15.72:0/3462274508 <== mgr.14152 v2:172.21.15.72:6800/3348457169 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ffa08062e80 con 0x7ff9f00a3c40 2024-04-25T13:12:45.014 INFO:teuthology.orchestra.run.smithi072.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T13:12:45.014 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V7400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T13:12:45.014 INFO:teuthology.orchestra.run.smithi072.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:12:45.014 INFO:teuthology.orchestra.run.smithi072.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:12:45.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9f00a3c40 msgr2=0x7ff9f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:45.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9f00a3c40 0x7ff9f00a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa08103b00 tx=0x7ff9f800a3b0 comp rx=0 tx=0).stop 2024-04-25T13:12:45.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 msgr2=0x7ffa08074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:45.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 0x7ffa08074e30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc003790 tx=0x7ff9fc005ef0 comp rx=0 tx=0).stop 2024-04-25T13:12:45.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 shutdown_connections 2024-04-25T13:12:45.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa08104640 0x7ffa08075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:45.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa08102aa0 0x7ffa080767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7ff9f00a3c40 0x7ff9f00a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 --2- 172.21.15.72:0/3462274508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa08103ca0 0x7ffa08074e30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 >> 172.21.15.72:0/3462274508 conn(0x7ffa080fe040 msgr2=0x7ffa08106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 shutdown_connections 2024-04-25T13:12:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:45.016+0000 7ffa0dada700 1 -- 172.21.15.72:0/3462274508 wait complete. 2024-04-25T13:12:45.942 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:12:46.203 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:12:46.204 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T13:12:46.460 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:46.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:46 smithi072 ceph-mon[29199]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:46.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:46 smithi072 ceph-mon[29199]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:12:46.533 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:46 smithi193 ceph-mon[32743]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:46.533 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:46 smithi193 ceph-mon[32743]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:12:46.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:46 smithi164 ceph-mon[32418]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:46.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:46 smithi164 ceph-mon[32418]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:12:48.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 -- 172.21.15.72:0/4033627605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c102aa0 msgr2=0x7fa06c102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 --2- 172.21.15.72:0/4033627605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c102aa0 0x7fa06c102ec0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0099c0 tx=0x7fa05c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 -- 172.21.15.72:0/4033627605 shutdown_connections 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 --2- 172.21.15.72:0/4033627605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa06c104640 0x7fa06c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 --2- 172.21.15.72:0/4033627605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c102aa0 0x7fa06c102ec0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 --2- 172.21.15.72:0/4033627605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa06c103ca0 0x7fa06c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 -- 172.21.15.72:0/4033627605 >> 172.21.15.72:0/4033627605 conn(0x7fa06c0fe040 msgr2=0x7fa06c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 -- 172.21.15.72:0/4033627605 shutdown_connections 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 -- 172.21.15.72:0/4033627605 wait complete. 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 Processor -- start 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 -- start start 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa06c102aa0 0x7fa06c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa06c103ca0 0x7fa06c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.239+0000 7fa071654700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c104640 0x7fa06c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa071654700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c10e610 con 0x7fa06c104640 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa071654700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c10e490 con 0x7fa06c102aa0 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa071654700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c10e790 con 0x7fa06c103ca0 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c104640 0x7fa06c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c104640 0x7fa06c1a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33812/0 (socket says 172.21.15.72:33812) 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06b7fe700 1 -- 172.21.15.72:0/1769503459 learned_addr learned my addr 172.21.15.72:0/1769503459 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06affd700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa06c102aa0 0x7fa06c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06a7fc700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa06c103ca0 0x7fa06c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06b7fe700 1 -- 172.21.15.72:0/1769503459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa06c103ca0 msgr2=0x7fa06c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06b7fe700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa06c103ca0 0x7fa06c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06b7fe700 1 -- 172.21.15.72:0/1769503459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa06c102aa0 msgr2=0x7fa06c198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06b7fe700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa06c102aa0 0x7fa06c198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06b7fe700 1 -- 172.21.15.72:0/1769503459 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c1a0970 con 0x7fa06c104640 2024-04-25T13:12:48.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06affd700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa06c102aa0 0x7fa06c198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:48.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa06b7fe700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c104640 0x7fa06c1a02a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b970 tx=0x7fa06400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:48.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa053fff700 1 -- 172.21.15.72:0/1769503459 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064019070 con 0x7fa06c104640 2024-04-25T13:12:48.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa053fff700 1 -- 172.21.15.72:0/1769503459 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa064014440 con 0x7fa06c104640 2024-04-25T13:12:48.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c1a0c60 con 0x7fa06c104640 2024-04-25T13:12:48.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa053fff700 1 -- 172.21.15.72:0/1769503459 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06400ce60 con 0x7fa06c104640 2024-04-25T13:12:48.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.240+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c1a1150 con 0x7fa06c104640 2024-04-25T13:12:48.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.241+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa06c069730 con 0x7fa06c104640 2024-04-25T13:12:48.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.242+0000 7fa053fff700 1 -- 172.21.15.72:0/1769503459 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa06401c070 con 0x7fa06c104640 2024-04-25T13:12:48.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.243+0000 7fa053fff700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0540a3c40 0x7fa0540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:48.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.243+0000 7fa06affd700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0540a3c40 0x7fa0540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:48.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.243+0000 7fa053fff700 1 -- 172.21.15.72:0/1769503459 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa0640cfd20 con 0x7fa06c104640 2024-04-25T13:12:48.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.243+0000 7fa06affd700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0540a3c40 0x7fa0540a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009990 tx=0x7fa05c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:48.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.247+0000 7fa053fff700 1 -- 172.21.15.72:0/1769503459 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa06409c6f0 con 0x7fa06c104640 2024-04-25T13:12:48.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.493+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa06c02df80 con 0x7fa06c104640 2024-04-25T13:12:48.493 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:48 smithi072 ceph-mon[29199]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.493+0000 7fa053fff700 1 -- 172.21.15.72:0/1769503459 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3056 (secure 0 0 0) 0x7fa06c02df80 con 0x7fa06c104640 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stdout:e20 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stdout:legacy client fscid: 1 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stdout:Filesystem 'cephfs' (1) 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stdout:fs_name cephfs 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 20 2024-04-25T13:12:48.494 INFO:teuthology.orchestra.run.smithi072.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:created 2024-04-25T13:11:04.601639+0000 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:modified 2024-04-25T13:11:09.292132+0000 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:tableserver 0 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:root 0 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:session_timeout 300 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:session_autoclose 300 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:max_file_size 1099511627776 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:max_xattr_size 65536 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:required_client_features {} 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:last_failure 0 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:last_failure_osd_epoch 0 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:max_mds 3 2024-04-25T13:12:48.495 INFO:teuthology.orchestra.run.smithi072.stdout:in 0,1,2 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:up {0=24509,1=24493,2=24478} 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:failed 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:damaged 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:stopped 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:data_pools [3] 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:metadata_pool 2 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:inline_data disabled 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:balancer 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:bal_rank_mask -1 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:standby_count_wanted 0 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:qdb_cluster leader: 24509 members: 24478,24493,24509 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b{0:24509} state up:active seq 3 addr [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.h{0:24476} state up:standby-replay seq 1 addr [v2:172.21.15.164:6836/1335235896,v1:172.21.15.164:6837/1335235896] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.i{1:24493} state up:active seq 6 addr [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.f{1:24466} state up:standby-replay seq 1 addr [v2:172.21.15.193:6834/1113925217,v1:172.21.15.193:6835/1113925217] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c{2:24478} state up:active seq 7 addr [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.e{2:24443} state up:standby-replay seq 1 addr [v2:172.21.15.164:6834/3356589609,v1:172.21.15.164:6835/3356589609] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout:Standby daemons: 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.j{-1:14532} state up:standby seq 1 addr [v2:172.21.15.72:6834/3145624420,v1:172.21.15.72:6835/3145624420] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.72:6836/1560520314,v1:172.21.15.72:6837/1560520314] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.g{-1:14616} state up:standby seq 1 addr [v2:172.21.15.72:6838/3204730698,v1:172.21.15.72:6839/3204730698] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a{-1:14628} state up:standby seq 1 addr [v2:172.21.15.72:6840/882585461,v1:172.21.15.72:6841/882585461] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.k{-1:24410} state up:standby seq 1 addr [v2:172.21.15.164:6832/1516437385,v1:172.21.15.164:6833/1516437385] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.497 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.193:6832/3064383723,v1:172.21.15.193:6833/3064383723] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:12:48.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0540a3c40 msgr2=0x7fa0540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:48.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0540a3c40 0x7fa0540a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009990 tx=0x7fa05c005fb0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c104640 msgr2=0x7fa06c1a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:48.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c104640 0x7fa06c1a02a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b970 tx=0x7fa06400bd30 comp rx=0 tx=0).stop 2024-04-25T13:12:48.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 shutdown_connections 2024-04-25T13:12:48.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0540a3c40 0x7fa0540a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa06c104640 0x7fa06c1a02a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa06c102aa0 0x7fa06c198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 --2- 172.21.15.72:0/1769503459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa06c103ca0 0x7fa06c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:48.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 >> 172.21.15.72:0/1769503459 conn(0x7fa06c0fe040 msgr2=0x7fa06c107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:48.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 shutdown_connections 2024-04-25T13:12:48.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:48.496+0000 7fa071654700 1 -- 172.21.15.72:0/1769503459 wait complete. 2024-04-25T13:12:48.499 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 20 2024-04-25T13:12:48.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:48 smithi164 ceph-mon[32418]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T13:12:48.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:48 smithi193 ceph-mon[32743]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T13:12:49.234 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T13:12:49.484 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:49.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:49 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1769503459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:12:49.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:49 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1769503459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:12:49.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:49 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1769503459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:12:50.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:50 smithi072 ceph-mon[29199]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:12:50.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:50 smithi164 ceph-mon[32418]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:12:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:50 smithi193 ceph-mon[32743]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:12:51.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.116+0000 7f4ed40e6700 1 -- 172.21.15.72:0/1541721051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc101b90 msgr2=0x7f4ecc102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:51.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.116+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/1541721051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc101b90 0x7f4ecc102010 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009990 tx=0x7f4ebc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 -- 172.21.15.72:0/1541721051 shutdown_connections 2024-04-25T13:12:51.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/1541721051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ecc102550 0x7f4ecc106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/1541721051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ecc100e20 0x7f4ecc101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/1541721051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc101b90 0x7f4ecc102010 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 -- 172.21.15.72:0/1541721051 >> 172.21.15.72:0/1541721051 conn(0x7f4ecc0fc780 msgr2=0x7f4ecc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:51.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 -- 172.21.15.72:0/1541721051 shutdown_connections 2024-04-25T13:12:51.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 -- 172.21.15.72:0/1541721051 wait complete. 2024-04-25T13:12:51.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 Processor -- start 2024-04-25T13:12:51.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 -- start start 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ecc100e20 0x7f4ecc19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ecc101b90 0x7f4ecc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc102550 0x7f4ecc1a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc114c40 con 0x7f4ecc102550 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc114ac0 con 0x7f4ecc100e20 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.117+0000 7f4ed40e6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc114dc0 con 0x7f4ecc101b90 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed2683700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc102550 0x7f4ecc1a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed2683700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc102550 0x7f4ecc1a67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33832/0 (socket says 172.21.15.72:33832) 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed2683700 1 -- 172.21.15.72:0/4122453566 learned_addr learned my addr 172.21.15.72:0/4122453566 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed1e82700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ecc100e20 0x7f4ecc19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed1681700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ecc101b90 0x7f4ecc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed2683700 1 -- 172.21.15.72:0/4122453566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ecc101b90 msgr2=0x7f4ecc19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:51.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed2683700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ecc101b90 0x7f4ecc19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed2683700 1 -- 172.21.15.72:0/4122453566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ecc100e20 msgr2=0x7f4ecc19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:51.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed2683700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ecc100e20 0x7f4ecc19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed2683700 1 -- 172.21.15.72:0/4122453566 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc1a6e70 con 0x7f4ecc102550 2024-04-25T13:12:51.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed2683700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc102550 0x7f4ecc1a67d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4007da0 tx=0x7f4ec400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:51.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ec2ffd700 1 -- 172.21.15.72:0/4122453566 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4016020 con 0x7f4ecc102550 2024-04-25T13:12:51.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc1a7160 con 0x7f4ecc102550 2024-04-25T13:12:51.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ec2ffd700 1 -- 172.21.15.72:0/4122453566 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ec400ad00 con 0x7f4ecc102550 2024-04-25T13:12:51.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ec2ffd700 1 -- 172.21.15.72:0/4122453566 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40115e0 con 0x7f4ecc102550 2024-04-25T13:12:51.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.118+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc1a7650 con 0x7f4ecc102550 2024-04-25T13:12:51.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.119+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc069730 con 0x7f4ecc102550 2024-04-25T13:12:51.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.120+0000 7f4ec2ffd700 1 -- 172.21.15.72:0/4122453566 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ec4019020 con 0x7f4ecc102550 2024-04-25T13:12:51.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.121+0000 7f4ec2ffd700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4eb80a3c40 0x7f4eb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:51.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.121+0000 7f4ed1e82700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4eb80a3c40 0x7f4eb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:51.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.121+0000 7f4ec2ffd700 1 -- 172.21.15.72:0/4122453566 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f4ec40d0810 con 0x7f4ecc102550 2024-04-25T13:12:51.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.121+0000 7f4ed1e82700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4eb80a3c40 0x7f4eb80a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8004630 tx=0x7f4ec800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:51.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.124+0000 7f4ec2ffd700 1 -- 172.21.15.72:0/4122453566 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ec4020020 con 0x7f4ecc102550 2024-04-25T13:12:51.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.327+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f4ecc02df80 con 0x7f4ecc102550 2024-04-25T13:12:51.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.328+0000 7f4ec2ffd700 1 -- 172.21.15.72:0/4122453566 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5320 (secure 0 0 0) 0x7f4ec409d1e0 con 0x7f4ecc102550 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 90 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:fsid d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:created 2024-04-25T13:02:57.590034+0000 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:modified 2024-04-25T13:12:08.140624+0000 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:crush_version 26 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:full_ratio 0.95 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:backfillfull_ratio 0.9 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:nearfull_ratio 0.85 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:require_min_compat_client luminous 2024-04-25T13:12:51.328 INFO:teuthology.orchestra.run.smithi072.stdout:min_compat_client luminous 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:require_osd_release squid 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:stretch_mode_enabled false 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:max_osd 12 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6802/2197040660,v1:172.21.15.72:6803/2197040660] [v2:172.21.15.72:6804/2197040660,v1:172.21.15.72:6805/2197040660] exists,up 86128e0d-c8b1-4771-a222-d42316fe3351 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/4084359931,v1:172.21.15.164:6801/4084359931] [v2:172.21.15.164:6802/4084359931,v1:172.21.15.164:6803/4084359931] exists,up a2389db1-f136-4e7a-a5c9-bc63d97b938c 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/1367843850,v1:172.21.15.193:6801/1367843850] [v2:172.21.15.193:6802/1367843850,v1:172.21.15.193:6803/1367843850] exists,up 778a3d43-534a-4aca-b391-07b8b9e045a4 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:osd.3 up in weight 1 up_from 25 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6810/840491303,v1:172.21.15.72:6811/840491303] [v2:172.21.15.72:6812/840491303,v1:172.21.15.72:6813/840491303] exists,up 2134ce99-2a05-49fd-8653-81180a5c199a 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:osd.4 up in weight 1 up_from 33 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/1819655779,v1:172.21.15.164:6809/1819655779] [v2:172.21.15.164:6810/1819655779,v1:172.21.15.164:6811/1819655779] exists,up 79d3bd39-e0cb-403c-8911-1a633198aa3d 2024-04-25T13:12:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:osd.5 up in weight 1 up_from 39 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/2502247683,v1:172.21.15.193:6809/2502247683] [v2:172.21.15.193:6810/2502247683,v1:172.21.15.193:6811/2502247683] exists,up c64e72ba-dfe6-43fe-ba1c-4819ac7bc211 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:osd.6 up in weight 1 up_from 47 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6818/3792057765,v1:172.21.15.72:6819/3792057765] [v2:172.21.15.72:6820/3792057765,v1:172.21.15.72:6821/3792057765] exists,up e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:osd.7 up in weight 1 up_from 52 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/3610097866,v1:172.21.15.164:6817/3610097866] [v2:172.21.15.164:6818/3610097866,v1:172.21.15.164:6819/3610097866] exists,up 7ca180ab-29ed-4109-bb1d-f7d19cb086b5 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:osd.8 up in weight 1 up_from 61 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/2354460844,v1:172.21.15.193:6817/2354460844] [v2:172.21.15.193:6818/2354460844,v1:172.21.15.193:6819/2354460844] exists,up dd29481d-1fbb-4b9b-b066-f85db8310162 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:osd.9 up in weight 1 up_from 69 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6826/4174455071,v1:172.21.15.72:6827/4174455071] [v2:172.21.15.72:6828/4174455071,v1:172.21.15.72:6829/4174455071] exists,up 10681718-08e5-4452-845f-75a2299d281f 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:osd.10 up in weight 1 up_from 75 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/1853012500,v1:172.21.15.164:6825/1853012500] [v2:172.21.15.164:6826/1853012500,v1:172.21.15.164:6827/1853012500] exists,up fcbcf574-0a98-411a-ac22-fe6520a5c1eb 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/2763093490,v1:172.21.15.193:6825/2763093490] [v2:172.21.15.193:6826/2763093490,v1:172.21.15.193:6827/2763093490] exists,up 36fabfaa-4071-45f7-8d95-1a989dcba56b 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.b [2,4] 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.d [5,11] 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.10 [8,11] 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.1a [8,1] 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.1d [2,4,7,6] 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.0 [4,8] 2024-04-25T13:12:51.330 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.1 [4,9] 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.1e [3,9] 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.21 [5,9] 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6801/2774697366 expires 2024-04-26T13:04:07.045745+0000 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6800/2774697366 expires 2024-04-26T13:04:07.045745+0000 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/1368623402 expires 2024-04-26T13:04:07.045745+0000 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/338238237 expires 2024-04-26T13:04:07.045745+0000 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/2198731116 expires 2024-04-26T13:03:39.743288+0000 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/830672933 expires 2024-04-26T13:03:39.743288+0000 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6800/2706752188 expires 2024-04-26T13:03:39.743288+0000 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/1451045393 expires 2024-04-26T13:03:39.743288+0000 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/1385198849 expires 2024-04-26T13:04:07.045745+0000 2024-04-25T13:12:51.331 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6801/2706752188 expires 2024-04-26T13:03:39.743288+0000 2024-04-25T13:12:51.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4eb80a3c40 msgr2=0x7f4eb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:51.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4eb80a3c40 0x7f4eb80a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8004630 tx=0x7f4ec800b3c0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc102550 msgr2=0x7f4ecc1a67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:51.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc102550 0x7f4ecc1a67d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4007da0 tx=0x7f4ec400a340 comp rx=0 tx=0).stop 2024-04-25T13:12:51.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 shutdown_connections 2024-04-25T13:12:51.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f4eb80a3c40 0x7f4eb80a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ecc102550 0x7f4ecc1a67d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ecc100e20 0x7f4ecc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 --2- 172.21.15.72:0/4122453566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ecc101b90 0x7f4ecc19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:51.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 >> 172.21.15.72:0/4122453566 conn(0x7f4ecc0fc780 msgr2=0x7f4ecc102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:51.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 shutdown_connections 2024-04-25T13:12:51.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:51.330+0000 7f4ed40e6700 1 -- 172.21.15.72:0/4122453566 wait complete. 2024-04-25T13:12:52.035 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T13:12:52.048 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:12:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.150+0000 7fedee19b640 1 -- 172.21.15.72:0/1390125846 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102040 msgr2=0x7fede81024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.150+0000 7fedee19b640 1 --2- 172.21.15.72:0/1390125846 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102040 0x7fede81024c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feddc009900 tx=0x7feddc02a410 comp rx=0 tx=0).stop 2024-04-25T13:12:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.150+0000 7fedee19b640 1 -- 172.21.15.72:0/1390125846 shutdown_connections 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.150+0000 7fedee19b640 1 --2- 172.21.15.72:0/1390125846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede8102a00 0x7fede8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.150+0000 7fedee19b640 1 --2- 172.21.15.72:0/1390125846 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102040 0x7fede81024c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.150+0000 7fedee19b640 1 --2- 172.21.15.72:0/1390125846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fede8100e60 0x7fede8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.150+0000 7fedee19b640 1 -- 172.21.15.72:0/1390125846 >> 172.21.15.72:0/1390125846 conn(0x7fede80fc440 msgr2=0x7fede80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.150+0000 7fedee19b640 1 -- 172.21.15.72:0/1390125846 shutdown_connections 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.150+0000 7fedee19b640 1 -- 172.21.15.72:0/1390125846 wait complete. 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fedee19b640 1 Processor -- start 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fedee19b640 1 -- start start 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fedee19b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fede8100e60 0x7fede8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fedee19b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede8102040 0x7fede81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fedee19b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102a00 0x7fede819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fedee19b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fede810ca00 con 0x7fede8102a00 2024-04-25T13:12:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fedee19b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fede810c880 con 0x7fede8100e60 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fedee19b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fede810cb80 con 0x7fede8102040 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102a00 0x7fede819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102a00 0x7fede819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33840/0 (socket says 172.21.15.72:33840) 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede7fff640 1 -- 172.21.15.72:0/530992171 learned_addr learned my addr 172.21.15.72:0/530992171 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede6ffd640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede8102040 0x7fede81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede7fff640 1 -- 172.21.15.72:0/530992171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede8102040 msgr2=0x7fede81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede7fff640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede8102040 0x7fede81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede7fff640 1 -- 172.21.15.72:0/530992171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fede8100e60 msgr2=0x7fede8198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede77fe640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fede8100e60 0x7fede8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede7fff640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fede8100e60 0x7fede8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.151+0000 7fede7fff640 1 -- 172.21.15.72:0/530992171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede819fe50 con 0x7fede8102a00 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.152+0000 7fede7fff640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102a00 0x7fede819f7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fede000cad0 tx=0x7fede000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.152+0000 7fede4ff9640 1 -- 172.21.15.72:0/530992171 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede000ae20 con 0x7fede8102a00 2024-04-25T13:12:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.152+0000 7fede4ff9640 1 -- 172.21.15.72:0/530992171 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede0005020 con 0x7fede8102a00 2024-04-25T13:12:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.152+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede81a0140 con 0x7fede8102a00 2024-04-25T13:12:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.152+0000 7fede4ff9640 1 -- 172.21.15.72:0/530992171 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede000f6b0 con 0x7fede8102a00 2024-04-25T13:12:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.152+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81a0690 con 0x7fede8102a00 2024-04-25T13:12:52.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.153+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede81a0950 con 0x7fede8102a00 2024-04-25T13:12:52.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.154+0000 7fede4ff9640 1 -- 172.21.15.72:0/530992171 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fede00051d0 con 0x7fede8102a00 2024-04-25T13:12:52.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.154+0000 7fede4ff9640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fedd00a3bb0 0x7fedd00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.155+0000 7fede77fe640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fedd00a3bb0 0x7fedd00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.155+0000 7fede4ff9640 1 -- 172.21.15.72:0/530992171 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fede00d04d0 con 0x7fede8102a00 2024-04-25T13:12:52.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.155+0000 7fede77fe640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fedd00a3bb0 0x7fedd00a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fedd8005fd0 tx=0x7fedd8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:52.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.159+0000 7fede4ff9640 1 -- 172.21.15.72:0/530992171 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fede0014030 con 0x7fede8102a00 2024-04-25T13:12:52.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.339+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fede81a0b20 con 0x7fede8102a00 2024-04-25T13:12:52.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.340+0000 7fede4ff9640 1 -- 172.21.15.72:0/530992171 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22740 (secure 0 0 0) 0x7fede009cea0 con 0x7fede8102a00 2024-04-25T13:12:52.340 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:52.341 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:12:08.140624+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:12:52.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fedd00a3bb0 msgr2=0x7fedd00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fedd00a3bb0 0x7fedd00a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fedd8005fd0 tx=0x7fedd8005850 comp rx=0 tx=0).stop 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102a00 msgr2=0x7fede819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102a00 0x7fede819f7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fede000cad0 tx=0x7fede000cf90 comp rx=0 tx=0).stop 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 shutdown_connections 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fedd00a3bb0 0x7fedd00a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fede8102a00 0x7fede819f7e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede8102040 0x7fede81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 --2- 172.21.15.72:0/530992171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fede8100e60 0x7fede8198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 >> 172.21.15.72:0/530992171 conn(0x7fede80fc440 msgr2=0x7fede80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 shutdown_connections 2024-04-25T13:12:52.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.342+0000 7fedee19b640 1 -- 172.21.15.72:0/530992171 wait complete. 2024-04-25T13:12:52.354 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:06:33.150709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:11:02.352089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:11:03.565033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:12:52.354 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:12:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 -- 172.21.15.72:0/2100677521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0103b50 msgr2=0x7f7ee010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/2100677521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0103b50 0x7f7ee010a3e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009ca0 tx=0x7f7ed802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 -- 172.21.15.72:0/2100677521 shutdown_connections 2024-04-25T13:12:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/2100677521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0103b50 0x7f7ee010a3e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/2100677521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ee0103190 0x7f7ee0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/2100677521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee0101fa0 0x7f7ee01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 -- 172.21.15.72:0/2100677521 >> 172.21.15.72:0/2100677521 conn(0x7f7ee00fd530 msgr2=0x7f7ee00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 -- 172.21.15.72:0/2100677521 shutdown_connections 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 -- 172.21.15.72:0/2100677521 wait complete. 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 Processor -- start 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.455+0000 7f7ee48e8640 1 -- start start 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ee48e8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0101fa0 0x7f7ee0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ee48e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee0103190 0x7f7ee01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ee48e8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ee0103b50 0x7f7ee019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ee48e8640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee010db20 con 0x7f7ee0101fa0 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ee48e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee010d9a0 con 0x7f7ee0103190 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ee48e8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee010dca0 con 0x7f7ee0103b50 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ede59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0101fa0 0x7f7ee0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ede59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0101fa0 0x7f7ee0198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33858/0 (socket says 172.21.15.72:33858) 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ede59c640 1 -- 172.21.15.72:0/1608441308 learned_addr learned my addr 172.21.15.72:0/1608441308 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:52.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7eddd9b640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee0103190 0x7f7ee01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7eded9d640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ee0103b50 0x7f7ee019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ede59c640 1 -- 172.21.15.72:0/1608441308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ee0103b50 msgr2=0x7f7ee019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ede59c640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ee0103b50 0x7f7ee019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ede59c640 1 -- 172.21.15.72:0/1608441308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee0103190 msgr2=0x7f7ee01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ede59c640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee0103190 0x7f7ee01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ede59c640 1 -- 172.21.15.72:0/1608441308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee019fdb0 con 0x7f7ee0101fa0 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7eded9d640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ee0103b50 0x7f7ee019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ede59c640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0101fa0 0x7f7ee0198190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000c970 tx=0x7f7ed000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ed77fe640 1 -- 172.21.15.72:0/1608441308 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0007c10 con 0x7f7ee0101fa0 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ed77fe640 1 -- 172.21.15.72:0/1608441308 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed0007db0 con 0x7f7ee0101fa0 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee01a00a0 con 0x7f7ee0101fa0 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ed77fe640 1 -- 172.21.15.72:0/1608441308 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed000f870 con 0x7f7ee0101fa0 2024-04-25T13:12:52.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.456+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee01a0670 con 0x7f7ee0101fa0 2024-04-25T13:12:52.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.457+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee01092c0 con 0x7f7ee0101fa0 2024-04-25T13:12:52.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.459+0000 7f7ed77fe640 1 -- 172.21.15.72:0/1608441308 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ed000fa10 con 0x7f7ee0101fa0 2024-04-25T13:12:52.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.459+0000 7f7ed77fe640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7ecc0a3bb0 0x7f7ecc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.459+0000 7f7eddd9b640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7ecc0a3bb0 0x7f7ecc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.459+0000 7f7ed77fe640 1 -- 172.21.15.72:0/1608441308 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f7ed00d0850 con 0x7f7ee0101fa0 2024-04-25T13:12:52.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.460+0000 7f7eddd9b640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7ecc0a3bb0 0x7f7ecc0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ee01996a0 tx=0x7f7ec8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:52.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.464+0000 7f7ed77fe640 1 -- 172.21.15.72:0/1608441308 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ed009d2b0 con 0x7f7ee0101fa0 2024-04-25T13:12:52.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:52 smithi072 ceph-mon[29199]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:12:52.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:52 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/4122453566' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:12:52.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:52 smithi164 ceph-mon[32418]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:12:52.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:52 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/4122453566' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:12:52.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.640+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ee0062b50 con 0x7f7ee0101fa0 2024-04-25T13:12:52.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.640+0000 7f7ed77fe640 1 -- 172.21.15.72:0/1608441308 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f7ed009cbb0 con 0x7f7ee0101fa0 2024-04-25T13:12:52.641 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-25T13:12:52.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7ecc0a3bb0 msgr2=0x7f7ecc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7ecc0a3bb0 0x7f7ecc0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ee01996a0 tx=0x7f7ec8008040 comp rx=0 tx=0).stop 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0101fa0 msgr2=0x7f7ee0198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0101fa0 0x7f7ee0198190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000c970 tx=0x7f7ed000ce30 comp rx=0 tx=0).stop 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 shutdown_connections 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f7ecc0a3bb0 0x7f7ecc0a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ee0103b50 0x7f7ee019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee0103190 0x7f7ee01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 --2- 172.21.15.72:0/1608441308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ee0101fa0 0x7f7ee0198190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 >> 172.21.15.72:0/1608441308 conn(0x7f7ee00fd530 msgr2=0x7f7ee0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 shutdown_connections 2024-04-25T13:12:52.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.642+0000 7f7ee48e8640 1 -- 172.21.15.72:0/1608441308 wait complete. 2024-04-25T13:12:52.654 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:12:52.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:52 smithi193 ceph-mon[32743]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:12:52.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:52 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/4122453566' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.754+0000 7fd1e023c640 1 -- 172.21.15.72:0/2623426103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8101fc0 msgr2=0x7fd1d8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.754+0000 7fd1e023c640 1 --2- 172.21.15.72:0/2623426103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8101fc0 0x7fd1d8102440 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009900 tx=0x7fd1c802a410 comp rx=0 tx=0).stop 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.754+0000 7fd1e023c640 1 -- 172.21.15.72:0/2623426103 shutdown_connections 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.754+0000 7fd1e023c640 1 --2- 172.21.15.72:0/2623426103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1d8102980 0x7fd1d81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.754+0000 7fd1e023c640 1 --2- 172.21.15.72:0/2623426103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8101fc0 0x7fd1d8102440 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.754+0000 7fd1e023c640 1 --2- 172.21.15.72:0/2623426103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d8100e70 0x7fd1d8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.754+0000 7fd1e023c640 1 -- 172.21.15.72:0/2623426103 >> 172.21.15.72:0/2623426103 conn(0x7fd1d80fc3e0 msgr2=0x7fd1d80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 -- 172.21.15.72:0/2623426103 shutdown_connections 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 -- 172.21.15.72:0/2623426103 wait complete. 2024-04-25T13:12:52.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 Processor -- start 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 -- start start 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8100e70 0x7fd1d8069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1d8101fc0 0x7fd1d819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d8102980 0x7fd1d81a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d810c9f0 con 0x7fd1d8100e70 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d810c870 con 0x7fd1d8102980 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1e023c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d810cb70 con 0x7fd1d8101fc0 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1ddfd8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8100e70 0x7fd1d8069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1ddfd8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8100e70 0x7fd1d8069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33872/0 (socket says 172.21.15.72:33872) 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.755+0000 7fd1ddfd8640 1 -- 172.21.15.72:0/3695630903 learned_addr learned my addr 172.21.15.72:0/3695630903 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:52.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1de7d9640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d8102980 0x7fd1d81a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1dd7d7640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1d8101fc0 0x7fd1d819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1ddfd8640 1 -- 172.21.15.72:0/3695630903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1d8101fc0 msgr2=0x7fd1d819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1ddfd8640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1d8101fc0 0x7fd1d819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1ddfd8640 1 -- 172.21.15.72:0/3695630903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d8102980 msgr2=0x7fd1d81a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1ddfd8640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d8102980 0x7fd1d81a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1ddfd8640 1 -- 172.21.15.72:0/3695630903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d81a1f80 con 0x7fd1d8100e70 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1de7d9640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d8102980 0x7fd1d81a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1ddfd8640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8100e70 0x7fd1d8069c70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400d950 tx=0x7fd1d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1ceffd640 1 -- 172.21.15.72:0/3695630903 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4017070 con 0x7fd1d8100e70 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d81a2270 con 0x7fd1d8100e70 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1ceffd640 1 -- 172.21.15.72:0/3695630903 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1d4009c30 con 0x7fd1d8100e70 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1ceffd640 1 -- 172.21.15.72:0/3695630903 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400f620 con 0x7fd1d8100e70 2024-04-25T13:12:52.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.756+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d81a2790 con 0x7fd1d8100e70 2024-04-25T13:12:52.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.757+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d8108180 con 0x7fd1d8100e70 2024-04-25T13:12:52.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.758+0000 7fd1ceffd640 1 -- 172.21.15.72:0/3695630903 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd1d4010460 con 0x7fd1d8100e70 2024-04-25T13:12:52.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.758+0000 7fd1ceffd640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd1c40a3bb0 0x7fd1c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:52.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.759+0000 7fd1ceffd640 1 -- 172.21.15.72:0/3695630903 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fd1d40d00f0 con 0x7fd1d8100e70 2024-04-25T13:12:52.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.759+0000 7fd1dd7d7640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd1c40a3bb0 0x7fd1c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:52.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.759+0000 7fd1dd7d7640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd1c40a3bb0 0x7fd1c40a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8006010 tx=0x7fd1c80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:52.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.763+0000 7fd1ceffd640 1 -- 172.21.15.72:0/3695630903 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1d409cac0 con 0x7fd1d8100e70 2024-04-25T13:12:52.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.938+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd1d8062b50 con 0x7fd1d8100e70 2024-04-25T13:12:52.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.938+0000 7fd1ceffd640 1 -- 172.21.15.72:0/3695630903 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fd1d4009de0 con 0x7fd1d8100e70 2024-04-25T13:12:52.939 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd1c40a3bb0 msgr2=0x7fd1c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd1c40a3bb0 0x7fd1c40a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8006010 tx=0x7fd1c80314a0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8100e70 msgr2=0x7fd1d8069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8100e70 0x7fd1d8069c70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400d950 tx=0x7fd1d400de10 comp rx=0 tx=0).stop 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 shutdown_connections 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fd1c40a3bb0 0x7fd1c40a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d8102980 0x7fd1d81a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1d8101fc0 0x7fd1d819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 --2- 172.21.15.72:0/3695630903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8100e70 0x7fd1d8069c70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:52.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 >> 172.21.15.72:0/3695630903 conn(0x7fd1d80fc3e0 msgr2=0x7fd1d8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:52.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 shutdown_connections 2024-04-25T13:12:52.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:52.940+0000 7fd1e023c640 1 -- 172.21.15.72:0/3695630903 wait complete. 2024-04-25T13:12:52.952 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:12:53.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.052+0000 7f3778f2d640 1 -- 172.21.15.72:0/2974783521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774100e90 msgr2=0x7f37741012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.052+0000 7f3778f2d640 1 --2- 172.21.15.72:0/2974783521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774100e90 0x7f37741012b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3768009900 tx=0x7f376802a410 comp rx=0 tx=0).stop 2024-04-25T13:12:53.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 -- 172.21.15.72:0/2974783521 shutdown_connections 2024-04-25T13:12:53.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 --2- 172.21.15.72:0/2974783521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37741029a0 0x7f37741092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 --2- 172.21.15.72:0/2974783521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3774101fe0 0x7f3774102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 --2- 172.21.15.72:0/2974783521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774100e90 0x7f37741012b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 -- 172.21.15.72:0/2974783521 >> 172.21.15.72:0/2974783521 conn(0x7f37740fc400 msgr2=0x7f37740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 -- 172.21.15.72:0/2974783521 shutdown_connections 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 -- 172.21.15.72:0/2974783521 wait complete. 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 Processor -- start 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 -- start start 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3774100e90 0x7f3774198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774101fe0 0x7f3774198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37741029a0 0x7f377419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f377410db30 con 0x7f3774101fe0 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f377410d9b0 con 0x7f3774100e90 2024-04-25T13:12:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.053+0000 7f3778f2d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f377410dcb0 con 0x7f37741029a0 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3771d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774101fe0 0x7f3774198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3771d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774101fe0 0x7f3774198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33886/0 (socket says 172.21.15.72:33886) 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3771d9b640 1 -- 172.21.15.72:0/1120935090 learned_addr learned my addr 172.21.15.72:0/1120935090 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f377259c640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3774100e90 0x7f3774198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3772d9d640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37741029a0 0x7f377419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3771d9b640 1 -- 172.21.15.72:0/1120935090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37741029a0 msgr2=0x7f377419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3771d9b640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37741029a0 0x7f377419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3771d9b640 1 -- 172.21.15.72:0/1120935090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3774100e90 msgr2=0x7f3774198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3771d9b640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3774100e90 0x7f3774198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3771d9b640 1 -- 172.21.15.72:0/1120935090 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377419fe70 con 0x7f3774101fe0 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3772d9d640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37741029a0 0x7f377419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f377259c640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3774100e90 0x7f3774198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:53.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3771d9b640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774101fe0 0x7f3774198790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f376400d770 tx=0x7f376400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:53.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f37637fe640 1 -- 172.21.15.72:0/1120935090 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764017070 con 0x7f3774101fe0 2024-04-25T13:12:53.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37741a0160 con 0x7f3774101fe0 2024-04-25T13:12:53.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f37637fe640 1 -- 172.21.15.72:0/1120935090 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3764009c30 con 0x7f3774101fe0 2024-04-25T13:12:53.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f37637fe640 1 -- 172.21.15.72:0/1120935090 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764003cc0 con 0x7f3774101fe0 2024-04-25T13:12:53.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.054+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37741a0700 con 0x7f3774101fe0 2024-04-25T13:12:53.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.055+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37741082c0 con 0x7f3774101fe0 2024-04-25T13:12:53.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.056+0000 7f37637fe640 1 -- 172.21.15.72:0/1120935090 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3764003e60 con 0x7f3774101fe0 2024-04-25T13:12:53.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.057+0000 7f37637fe640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f375c0a3bb0 0x7f375c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.057+0000 7f377259c640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f375c0a3bb0 0x7f375c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.057+0000 7f37637fe640 1 -- 172.21.15.72:0/1120935090 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f37640cfe50 con 0x7f3774101fe0 2024-04-25T13:12:53.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.058+0000 7f377259c640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f375c0a3bb0 0x7f375c0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3768006010 tx=0x7f3768000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:53.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.062+0000 7f37637fe640 1 -- 172.21.15.72:0/1120935090 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f376409c820 con 0x7f3774101fe0 2024-04-25T13:12:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.240+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3774062b50 con 0x7f3774101fe0 2024-04-25T13:12:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.240+0000 7f37637fe640 1 -- 172.21.15.72:0/1120935090 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f3764009de0 con 0x7f3774101fe0 2024-04-25T13:12:53.240 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f375c0a3bb0 msgr2=0x7f375c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f375c0a3bb0 0x7f375c0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3768006010 tx=0x7f3768000bc0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774101fe0 msgr2=0x7f3774198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774101fe0 0x7f3774198790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f376400d770 tx=0x7f376400dc30 comp rx=0 tx=0).stop 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 shutdown_connections 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f375c0a3bb0 0x7f375c0a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37741029a0 0x7f377419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3774101fe0 0x7f3774198790 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 --2- 172.21.15.72:0/1120935090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3774100e90 0x7f3774198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 >> 172.21.15.72:0/1120935090 conn(0x7f37740fc400 msgr2=0x7f3774105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 shutdown_connections 2024-04-25T13:12:53.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.242+0000 7f3778f2d640 1 -- 172.21.15.72:0/1120935090 wait complete. 2024-04-25T13:12:53.254 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:12:53.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 -- 172.21.15.72:0/3163257531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0941029b0 msgr2=0x7fc0941092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 --2- 172.21.15.72:0/3163257531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0941029b0 0x7fc0941092d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009ca0 tx=0x7fc08c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 -- 172.21.15.72:0/3163257531 shutdown_connections 2024-04-25T13:12:53.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 --2- 172.21.15.72:0/3163257531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0941029b0 0x7fc0941092d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 --2- 172.21.15.72:0/3163257531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc094101ff0 0x7fc094102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 --2- 172.21.15.72:0/3163257531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc094100ea0 0x7fc0941012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 -- 172.21.15.72:0/3163257531 >> 172.21.15.72:0/3163257531 conn(0x7fc0940fc430 msgr2=0x7fc0940fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 -- 172.21.15.72:0/3163257531 shutdown_connections 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 -- 172.21.15.72:0/3163257531 wait complete. 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 Processor -- start 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.352+0000 7fc09bde5640 1 -- start start 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc09bde5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc094100ea0 0x7fc094196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc09bde5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc094101ff0 0x7fc094197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc09bde5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0941029b0 0x7fc09419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc09bde5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410c9e0 con 0x7fc094101ff0 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc09bde5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410c860 con 0x7fc094100ea0 2024-04-25T13:12:53.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc09bde5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410cb60 con 0x7fc0941029b0 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099380640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc094101ff0 0x7fc094197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099380640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc094101ff0 0x7fc094197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33910/0 (socket says 172.21.15.72:33910) 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099380640 1 -- 172.21.15.72:0/1257269571 learned_addr learned my addr 172.21.15.72:0/1257269571 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099b81640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc094100ea0 0x7fc094196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099380640 1 -- 172.21.15.72:0/1257269571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0941029b0 msgr2=0x7fc09419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099380640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0941029b0 0x7fc09419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099380640 1 -- 172.21.15.72:0/1257269571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc094100ea0 msgr2=0x7fc094196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099380640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc094100ea0 0x7fc094196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099380640 1 -- 172.21.15.72:0/1257269571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09419ebe0 con 0x7fc094101ff0 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc099380640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc094101ff0 0x7fc094197500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc084007900 tx=0x7fc084007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc08affd640 1 -- 172.21.15.72:0/1257269571 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084019020 con 0x7fc094101ff0 2024-04-25T13:12:53.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc08affd640 1 -- 172.21.15.72:0/1257269571 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08400e6f0 con 0x7fc094101ff0 2024-04-25T13:12:53.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09406a610 con 0x7fc094101ff0 2024-04-25T13:12:53.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc08affd640 1 -- 172.21.15.72:0/1257269571 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084003870 con 0x7fc094101ff0 2024-04-25T13:12:53.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.353+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09406ab60 con 0x7fc094101ff0 2024-04-25T13:12:53.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.354+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0940691c0 con 0x7fc094101ff0 2024-04-25T13:12:53.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.355+0000 7fc08affd640 1 -- 172.21.15.72:0/1257269571 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc08400e8a0 con 0x7fc094101ff0 2024-04-25T13:12:53.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.356+0000 7fc08affd640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc0800a3bb0 0x7fc0800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.356+0000 7fc099b81640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc0800a3bb0 0x7fc0800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.356+0000 7fc08affd640 1 -- 172.21.15.72:0/1257269571 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fc0840d1d50 con 0x7fc094101ff0 2024-04-25T13:12:53.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.356+0000 7fc099b81640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc0800a3bb0 0x7fc0800a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc090005e10 tx=0x7fc090005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:53.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.359+0000 7fc08affd640 1 -- 172.21.15.72:0/1257269571 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc084023020 con 0x7fc094101ff0 2024-04-25T13:12:53.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:53 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/530992171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:53.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:53 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1608441308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:53.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:53 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3695630903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:53.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:53 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1120935090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:53 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/530992171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:53 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1608441308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:53 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3695630903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:53.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:53 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1120935090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:53.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.576+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc094107ce0 con 0x7fc094101ff0 2024-04-25T13:12:53.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.576+0000 7fc08affd640 1 -- 172.21.15.72:0/1257269571 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11881 (secure 0 0 0) 0x7fc08409e7b0 con 0x7fc094101ff0 2024-04-25T13:12:53.576 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:53.577 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:11:04.601639+0000","modified":"2024-04-25T13:11:09.292132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24493,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6839/1444196308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1444196308},{"type":"v1","addr":"172.21.15.164:6839","nonce":1444196308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24478,24493,24509]},"id":1}]} 2024-04-25T13:12:53.577 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 20 2024-04-25T13:12:53.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc0800a3bb0 msgr2=0x7fc0800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc0800a3bb0 0x7fc0800a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc090005e10 tx=0x7fc090005da0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc094101ff0 msgr2=0x7fc094197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc094101ff0 0x7fc094197500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc084007900 tx=0x7fc084007dc0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 shutdown_connections 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc0800a3bb0 0x7fc0800a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0941029b0 0x7fc09419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc094101ff0 0x7fc094197500 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 --2- 172.21.15.72:0/1257269571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc094100ea0 0x7fc094196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 >> 172.21.15.72:0/1257269571 conn(0x7fc0940fc430 msgr2=0x7fc0941052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 shutdown_connections 2024-04-25T13:12:53.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.578+0000 7fc09bde5640 1 -- 172.21.15.72:0/1257269571 wait complete. 2024-04-25T13:12:53.591 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.688+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/1029938330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 msgr2=0x7f8a081012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.688+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/1029938330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 0x7f8a081012c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009900 tx=0x7f89fc02a410 comp rx=0 tx=0).stop 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.688+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/1029938330 shutdown_connections 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.688+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/1029938330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a081029b0 0x7f8a081092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.688+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/1029938330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a08101ff0 0x7f8a08102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.688+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/1029938330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 0x7f8a081012c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.688+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/1029938330 >> 172.21.15.72:0/1029938330 conn(0x7f8a080fc430 msgr2=0x7f8a080fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/1029938330 shutdown_connections 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/1029938330 wait complete. 2024-04-25T13:12:53.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 Processor -- start 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 -- start start 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 0x7f8a08198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a08101ff0 0x7f8a081987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a081029b0 0x7f8a0819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0810c9f0 con 0x7f8a08100ea0 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0810c870 con 0x7f8a08101ff0 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a0ddbf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0810cb70 con 0x7f8a081029b0 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a077fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 0x7f8a08198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a077fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 0x7f8a08198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33930/0 (socket says 172.21.15.72:33930) 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.689+0000 7f8a077fe640 1 -- 172.21.15.72:0/2463233600 learned_addr learned my addr 172.21.15.72:0/2463233600 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a06ffd640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a08101ff0 0x7f8a081987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a07fff640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a081029b0 0x7f8a0819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a077fe640 1 -- 172.21.15.72:0/2463233600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a081029b0 msgr2=0x7f8a0819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a077fe640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a081029b0 0x7f8a0819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a077fe640 1 -- 172.21.15.72:0/2463233600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a08101ff0 msgr2=0x7f8a081987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a077fe640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a08101ff0 0x7f8a081987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a077fe640 1 -- 172.21.15.72:0/2463233600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0819fe30 con 0x7f8a08100ea0 2024-04-25T13:12:53.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a077fe640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 0x7f8a08198260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f89fc005940 tx=0x7f89fc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:53.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a04ff9640 1 -- 172.21.15.72:0/2463233600 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc03f030 con 0x7f8a08100ea0 2024-04-25T13:12:53.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a081a00c0 con 0x7f8a08100ea0 2024-04-25T13:12:53.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a04ff9640 1 -- 172.21.15.72:0/2463233600 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89fc003cb0 con 0x7f8a08100ea0 2024-04-25T13:12:53.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a04ff9640 1 -- 172.21.15.72:0/2463233600 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc035070 con 0x7f8a08100ea0 2024-04-25T13:12:53.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.690+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081a05b0 con 0x7f8a08100ea0 2024-04-25T13:12:53.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.691+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a081a0810 con 0x7f8a08100ea0 2024-04-25T13:12:53.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.692+0000 7f8a04ff9640 1 -- 172.21.15.72:0/2463233600 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f89fc042030 con 0x7f8a08100ea0 2024-04-25T13:12:53.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.693+0000 7f8a04ff9640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f89f00a3bb0 0x7f89f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.693+0000 7f8a06ffd640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f89f00a3bb0 0x7f89f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.693+0000 7f8a04ff9640 1 -- 172.21.15.72:0/2463233600 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f89fc0f0d60 con 0x7f8a08100ea0 2024-04-25T13:12:53.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.693+0000 7f8a06ffd640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f89f00a3bb0 0x7f89f00a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a08199720 tx=0x7f89f800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:53.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.697+0000 7f8a04ff9640 1 -- 172.21.15.72:0/2463233600 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89fc003e60 con 0x7f8a08100ea0 2024-04-25T13:12:53.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:53 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/530992171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:53.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:53 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1608441308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:53.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:53 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3695630903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:53.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:53 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1120935090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:53.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.876+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a081a09e0 con 0x7f8a08100ea0 2024-04-25T13:12:53.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.877+0000 7f8a04ff9640 1 -- 172.21.15.72:0/2463233600 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22740 (secure 0 0 0) 0x7f89fc0bd730 con 0x7f8a08100ea0 2024-04-25T13:12:53.878 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:53.878 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:12:08.140624+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.879+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f89f00a3bb0 msgr2=0x7f89f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.879+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f89f00a3bb0 0x7f89f00a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a08199720 tx=0x7f89f800b390 comp rx=0 tx=0).stop 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.879+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 msgr2=0x7f8a08198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.879+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 0x7f8a08198260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f89fc005940 tx=0x7f89fc004430 comp rx=0 tx=0).stop 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.880+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 shutdown_connections 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.880+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f89f00a3bb0 0x7f89f00a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.880+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a081029b0 0x7f8a0819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.880+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a08101ff0 0x7f8a081987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.880+0000 7f8a0ddbf640 1 --2- 172.21.15.72:0/2463233600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a08100ea0 0x7f8a08198260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.880+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 >> 172.21.15.72:0/2463233600 conn(0x7f8a080fc430 msgr2=0x7f8a0806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.880+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 shutdown_connections 2024-04-25T13:12:53.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.880+0000 7f8a0ddbf640 1 -- 172.21.15.72:0/2463233600 wait complete. 2024-04-25T13:12:53.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:06:33.150709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:11:02.352089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:11:03.565033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:12:53.893 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:12:53.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.993+0000 7f3ad6e60640 1 -- 172.21.15.72:0/3484006791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0103190 msgr2=0x7f3ad0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.993+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/3484006791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0103190 0x7f3ad0103610 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009900 tx=0x7f3ac402a410 comp rx=0 tx=0).stop 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.993+0000 7f3ad6e60640 1 -- 172.21.15.72:0/3484006791 shutdown_connections 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.993+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/3484006791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ad0103b50 0x7f3ad010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.993+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/3484006791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0103190 0x7f3ad0103610 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.993+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/3484006791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0101fa0 0x7f3ad01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.993+0000 7f3ad6e60640 1 -- 172.21.15.72:0/3484006791 >> 172.21.15.72:0/3484006791 conn(0x7f3ad00fd530 msgr2=0x7f3ad00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.993+0000 7f3ad6e60640 1 -- 172.21.15.72:0/3484006791 shutdown_connections 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.993+0000 7f3ad6e60640 1 -- 172.21.15.72:0/3484006791 wait complete. 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad6e60640 1 Processor -- start 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad6e60640 1 -- start start 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad6e60640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0101fa0 0x7f3ad0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad6e60640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0103190 0x7f3ad01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad6e60640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ad0103b50 0x7f3ad019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad6e60640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad010db90 con 0x7f3ad0101fa0 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad6e60640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad010da10 con 0x7f3ad0103b50 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad6e60640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad010dd10 con 0x7f3ad0103190 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad4bfc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0101fa0 0x7f3ad0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad4bfc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0101fa0 0x7f3ad0198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33952/0 (socket says 172.21.15.72:33952) 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad4bfc640 1 -- 172.21.15.72:0/1189709238 learned_addr learned my addr 172.21.15.72:0/1189709238 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad4bfc640 1 -- 172.21.15.72:0/1189709238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0103190 msgr2=0x7f3ad01987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad4bfc640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0103190 0x7f3ad01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad4bfc640 1 -- 172.21.15.72:0/1189709238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ad0103b50 msgr2=0x7f3ad019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad4bfc640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ad0103b50 0x7f3ad019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.994+0000 7f3ad4bfc640 1 -- 172.21.15.72:0/1189709238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad019fe80 con 0x7f3ad0101fa0 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.995+0000 7f3ad4bfc640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0101fa0 0x7f3ad0198260 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000c970 tx=0x7f3ac000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:53.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.995+0000 7f3acdffb640 1 -- 172.21.15.72:0/1189709238 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0004220 con 0x7f3ad0101fa0 2024-04-25T13:12:53.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.995+0000 7f3acdffb640 1 -- 172.21.15.72:0/1189709238 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac00043c0 con 0x7f3ad0101fa0 2024-04-25T13:12:53.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.995+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad01a0170 con 0x7f3ad0101fa0 2024-04-25T13:12:53.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.995+0000 7f3acdffb640 1 -- 172.21.15.72:0/1189709238 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0004a00 con 0x7f3ad0101fa0 2024-04-25T13:12:53.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.995+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a06f0 con 0x7f3ad0101fa0 2024-04-25T13:12:53.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.996+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad0109390 con 0x7f3ad0101fa0 2024-04-25T13:12:53.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.997+0000 7f3acdffb640 1 -- 172.21.15.72:0/1189709238 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3ac00079b0 con 0x7f3ad0101fa0 2024-04-25T13:12:53.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.997+0000 7f3acdffb640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3ab80a3bb0 0x7f3ab80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:53.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.997+0000 7f3acdffb640 1 -- 172.21.15.72:0/1189709238 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f3ac00d0220 con 0x7f3ad0101fa0 2024-04-25T13:12:53.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.997+0000 7f3acffff640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3ab80a3bb0 0x7f3ab80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:53.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:53.998+0000 7f3acffff640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3ab80a3bb0 0x7f3ab80a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009230 tx=0x7f3ac4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:54.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.002+0000 7f3acdffb640 1 -- 172.21.15.72:0/1189709238 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ac0014030 con 0x7f3ad0101fa0 2024-04-25T13:12:54.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.181+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ad0062bf0 con 0x7f3ad0101fa0 2024-04-25T13:12:54.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.181+0000 7f3acdffb640 1 -- 172.21.15.72:0/1189709238 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f3ac009cbf0 con 0x7f3ad0101fa0 2024-04-25T13:12:54.182 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-25T13:12:54.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3ab80a3bb0 msgr2=0x7f3ab80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3ab80a3bb0 0x7f3ab80a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009230 tx=0x7f3ac4000bc0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0101fa0 msgr2=0x7f3ad0198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0101fa0 0x7f3ad0198260 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000c970 tx=0x7f3ac000ce30 comp rx=0 tx=0).stop 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 shutdown_connections 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3ab80a3bb0 0x7f3ab80a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ad0103b50 0x7f3ad019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0103190 0x7f3ad01987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 --2- 172.21.15.72:0/1189709238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0101fa0 0x7f3ad0198260 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 >> 172.21.15.72:0/1189709238 conn(0x7f3ad00fd530 msgr2=0x7f3ad0106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 shutdown_connections 2024-04-25T13:12:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.183+0000 7f3ad6e60640 1 -- 172.21.15.72:0/1189709238 wait complete. 2024-04-25T13:12:54.195 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:12:54.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 -- 172.21.15.72:0/1281395929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 msgr2=0x7f1dc410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/1281395929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 0x7f1dc410a0e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009ca0 tx=0x7f1dbc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 -- 172.21.15.72:0/1281395929 shutdown_connections 2024-04-25T13:12:54.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/1281395929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 0x7f1dc410a0e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/1281395929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dc4105360 0x7f1dc4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/1281395929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc4102a30 0x7f1dc4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 -- 172.21.15.72:0/1281395929 >> 172.21.15.72:0/1281395929 conn(0x7f1dc40fc440 msgr2=0x7f1dc40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 -- 172.21.15.72:0/1281395929 shutdown_connections 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 -- 172.21.15.72:0/1281395929 wait complete. 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 Processor -- start 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.296+0000 7f1dcc34b640 1 -- start start 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dcc34b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc4102a30 0x7f1dc41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dcc34b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dc4105360 0x7f1dc4198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dcc34b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 0x7f1dc419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dcc34b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc410dbd0 con 0x7f1dc4107c90 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dcc34b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc410da50 con 0x7f1dc4102a30 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dcc34b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc410dd50 con 0x7f1dc4105360 2024-04-25T13:12:54.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca8e8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 0x7f1dc419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca8e8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 0x7f1dc419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33978/0 (socket says 172.21.15.72:33978) 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca8e8640 1 -- 172.21.15.72:0/2604842714 learned_addr learned my addr 172.21.15.72:0/2604842714 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca0e7640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc4102a30 0x7f1dc41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dc98e6640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dc4105360 0x7f1dc4198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca8e8640 1 -- 172.21.15.72:0/2604842714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dc4105360 msgr2=0x7f1dc4198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca8e8640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dc4105360 0x7f1dc4198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca8e8640 1 -- 172.21.15.72:0/2604842714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc4102a30 msgr2=0x7f1dc41982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca8e8640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc4102a30 0x7f1dc41982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca8e8640 1 -- 172.21.15.72:0/2604842714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc419ff00 con 0x7f1dc4107c90 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca0e7640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc4102a30 0x7f1dc41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dca8e8640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 0x7f1dc419f890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc031a20 tx=0x7f1dbc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dbb7fe640 1 -- 172.21.15.72:0/2604842714 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc004240 con 0x7f1dc4107c90 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dbb7fe640 1 -- 172.21.15.72:0/2604842714 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dbc003e10 con 0x7f1dc4107c90 2024-04-25T13:12:54.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc41a0190 con 0x7f1dc4107c90 2024-04-25T13:12:54.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dbb7fe640 1 -- 172.21.15.72:0/2604842714 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0305c0 con 0x7f1dc4107c90 2024-04-25T13:12:54.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.297+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a0760 con 0x7f1dc4107c90 2024-04-25T13:12:54.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.298+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc40fdf30 con 0x7f1dc4107c90 2024-04-25T13:12:54.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.299+0000 7f1dbb7fe640 1 -- 172.21.15.72:0/2604842714 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1dbc02f660 con 0x7f1dc4107c90 2024-04-25T13:12:54.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.300+0000 7f1dbb7fe640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1db00a3bb0 0x7f1db00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.300+0000 7f1dca0e7640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1db00a3bb0 0x7f1db00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.300+0000 7f1dbb7fe640 1 -- 172.21.15.72:0/2604842714 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1dbc0f1ed0 con 0x7f1dc4107c90 2024-04-25T13:12:54.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.300+0000 7f1dca0e7640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1db00a3bb0 0x7f1db00a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009930 tx=0x7f1dc0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:54.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.304+0000 7f1dbb7fe640 1 -- 172.21.15.72:0/2604842714 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1dbc0be8a0 con 0x7f1dc4107c90 2024-04-25T13:12:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.479+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1dc4062b50 con 0x7f1dc4107c90 2024-04-25T13:12:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.480+0000 7f1dbb7fe640 1 -- 172.21.15.72:0/2604842714 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f1dbc0be1a0 con 0x7f1dc4107c90 2024-04-25T13:12:54.480 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-25T13:12:54.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1db00a3bb0 msgr2=0x7f1db00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1db00a3bb0 0x7f1db00a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009930 tx=0x7f1dc0008040 comp rx=0 tx=0).stop 2024-04-25T13:12:54.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 msgr2=0x7f1dc419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 0x7f1dc419f890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc031a20 tx=0x7f1dbc003c20 comp rx=0 tx=0).stop 2024-04-25T13:12:54.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 shutdown_connections 2024-04-25T13:12:54.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f1db00a3bb0 0x7f1db00a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1dc4107c90 0x7f1dc419f890 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dc4105360 0x7f1dc4198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 --2- 172.21.15.72:0/2604842714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc4102a30 0x7f1dc41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 >> 172.21.15.72:0/2604842714 conn(0x7f1dc40fc440 msgr2=0x7f1dc4103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:54.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 shutdown_connections 2024-04-25T13:12:54.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.482+0000 7f1dcc34b640 1 -- 172.21.15.72:0/2604842714 wait complete. 2024-04-25T13:12:54.492 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:54 smithi072 ceph-mon[29199]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:12:54.492 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:54 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1257269571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:54.492 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:54 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2463233600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:54.492 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:54 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1189709238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:54.493 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:12:54.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:54 smithi164 ceph-mon[32418]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:12:54.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:54 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1257269571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:54.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:54 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2463233600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:54.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:54 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1189709238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:54.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.593+0000 7f450002f640 1 -- 172.21.15.72:0/81271497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102040 msgr2=0x7f44f81024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.593+0000 7f450002f640 1 --2- 172.21.15.72:0/81271497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102040 0x7f44f81024c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009900 tx=0x7f44ec02a410 comp rx=0 tx=0).stop 2024-04-25T13:12:54.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.593+0000 7f450002f640 1 -- 172.21.15.72:0/81271497 shutdown_connections 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.593+0000 7f450002f640 1 --2- 172.21.15.72:0/81271497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44f8102a00 0x7f44f8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.593+0000 7f450002f640 1 --2- 172.21.15.72:0/81271497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102040 0x7f44f81024c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.593+0000 7f450002f640 1 --2- 172.21.15.72:0/81271497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f8100e60 0x7f44f8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.593+0000 7f450002f640 1 -- 172.21.15.72:0/81271497 >> 172.21.15.72:0/81271497 conn(0x7f44f80fc440 msgr2=0x7f44f80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.593+0000 7f450002f640 1 -- 172.21.15.72:0/81271497 shutdown_connections 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.593+0000 7f450002f640 1 -- 172.21.15.72:0/81271497 wait complete. 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f450002f640 1 Processor -- start 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f450002f640 1 -- start start 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f450002f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44f8100e60 0x7f44f81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f450002f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f8102040 0x7f44f8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f450002f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102a00 0x7f44f819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f450002f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f44f810ca10 con 0x7f44f8102a00 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f450002f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44f810c890 con 0x7f44f8100e60 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f450002f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44f810cb90 con 0x7f44f8102040 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fe5cc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102a00 0x7f44f819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fe5cc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102a00 0x7f44f819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33994/0 (socket says 172.21.15.72:33994) 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fe5cc640 1 -- 172.21.15.72:0/2453125863 learned_addr learned my addr 172.21.15.72:0/2453125863 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fd5ca640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f8102040 0x7f44f8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fe5cc640 1 -- 172.21.15.72:0/2453125863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f8102040 msgr2=0x7f44f8198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fe5cc640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f8102040 0x7f44f8198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fe5cc640 1 -- 172.21.15.72:0/2453125863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44f8100e60 msgr2=0x7f44f81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fddcb640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44f8100e60 0x7f44f81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fe5cc640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44f8100e60 0x7f44f81982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fe5cc640 1 -- 172.21.15.72:0/2453125863 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f819fe90 con 0x7f44f8102a00 2024-04-25T13:12:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.594+0000 7f44fe5cc640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102a00 0x7f44f819f820 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f44e800cad0 tx=0x7f44e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.595+0000 7f44f6ffd640 1 -- 172.21.15.72:0/2453125863 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e800ae20 con 0x7f44f8102a00 2024-04-25T13:12:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.595+0000 7f44f6ffd640 1 -- 172.21.15.72:0/2453125863 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e8005020 con 0x7f44f8102a00 2024-04-25T13:12:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.595+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f81a0180 con 0x7f44f8102a00 2024-04-25T13:12:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.595+0000 7f44f6ffd640 1 -- 172.21.15.72:0/2453125863 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e800f6b0 con 0x7f44f8102a00 2024-04-25T13:12:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.595+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f81a06d0 con 0x7f44f8102a00 2024-04-25T13:12:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.596+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f81a0990 con 0x7f44f8102a00 2024-04-25T13:12:54.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.597+0000 7f44f6ffd640 1 -- 172.21.15.72:0/2453125863 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f44e80051d0 con 0x7f44f8102a00 2024-04-25T13:12:54.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.597+0000 7f44f6ffd640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f44e00a3c30 0x7f44e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.597+0000 7f44fddcb640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f44e00a3c30 0x7f44e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.598+0000 7f44f6ffd640 1 -- 172.21.15.72:0/2453125863 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f44e80d0500 con 0x7f44f8102a00 2024-04-25T13:12:54.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.598+0000 7f44fddcb640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f44e00a3c30 0x7f44e00a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005fd0 tx=0x7f44f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:54.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.602+0000 7f44f6ffd640 1 -- 172.21.15.72:0/2453125863 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44e8014030 con 0x7f44f8102a00 2024-04-25T13:12:54.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:54 smithi193 ceph-mon[32743]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:12:54.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:54 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1257269571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:54.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:54 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2463233600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:54.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:54 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1189709238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:12:54.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.781+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f44f81a0b60 con 0x7f44f8102a00 2024-04-25T13:12:54.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.781+0000 7f44f6ffd640 1 -- 172.21.15.72:0/2453125863 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f44e809ced0 con 0x7f44f8102a00 2024-04-25T13:12:54.781 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f44e00a3c30 msgr2=0x7f44e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f44e00a3c30 0x7f44e00a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005fd0 tx=0x7f44f0005850 comp rx=0 tx=0).stop 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102a00 msgr2=0x7f44f819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102a00 0x7f44f819f820 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f44e800cad0 tx=0x7f44e800cf90 comp rx=0 tx=0).stop 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 shutdown_connections 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f44e00a3c30 0x7f44e00a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44f8102a00 0x7f44f819f820 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f8102040 0x7f44f8198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 --2- 172.21.15.72:0/2453125863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44f8100e60 0x7f44f81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.783+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 >> 172.21.15.72:0/2453125863 conn(0x7f44f80fc440 msgr2=0x7f44f806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.784+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 shutdown_connections 2024-04-25T13:12:54.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.784+0000 7f450002f640 1 -- 172.21.15.72:0/2453125863 wait complete. 2024-04-25T13:12:54.795 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.892+0000 7f6511dad640 1 -- 172.21.15.72:0/4193502178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 msgr2=0x7f650c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.892+0000 7f6511dad640 1 --2- 172.21.15.72:0/4193502178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 0x7f650c1092d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6504009ca0 tx=0x7f650402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.892+0000 7f6511dad640 1 -- 172.21.15.72:0/4193502178 shutdown_connections 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.892+0000 7f6511dad640 1 --2- 172.21.15.72:0/4193502178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 0x7f650c1092d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.892+0000 7f6511dad640 1 --2- 172.21.15.72:0/4193502178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f650c101ff0 0x7f650c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.892+0000 7f6511dad640 1 --2- 172.21.15.72:0/4193502178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650c100ea0 0x7f650c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.892+0000 7f6511dad640 1 -- 172.21.15.72:0/4193502178 >> 172.21.15.72:0/4193502178 conn(0x7f650c0fc430 msgr2=0x7f650c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.892+0000 7f6511dad640 1 -- 172.21.15.72:0/4193502178 shutdown_connections 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.892+0000 7f6511dad640 1 -- 172.21.15.72:0/4193502178 wait complete. 2024-04-25T13:12:54.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f6511dad640 1 Processor -- start 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f6511dad640 1 -- start start 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f6511dad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650c100ea0 0x7f650c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f6511dad640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f650c101ff0 0x7f650c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f6511dad640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 0x7f650c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f6511dad640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f650c10c9e0 con 0x7f650c1029b0 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f6511dad640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f650c10c860 con 0x7f650c100ea0 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f6511dad640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f650c10cb60 con 0x7f650c101ff0 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f650bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 0x7f650c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f650bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 0x7f650c19f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34012/0 (socket says 172.21.15.72:34012) 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f650bfff640 1 -- 172.21.15.72:0/2290178814 learned_addr learned my addr 172.21.15.72:0/2290178814 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.893+0000 7f650affd640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f650c101ff0 0x7f650c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f650b7fe640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650c100ea0 0x7f650c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f650bfff640 1 -- 172.21.15.72:0/2290178814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f650c101ff0 msgr2=0x7f650c19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f650bfff640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f650c101ff0 0x7f650c19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f650bfff640 1 -- 172.21.15.72:0/2290178814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650c100ea0 msgr2=0x7f650c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f650bfff640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650c100ea0 0x7f650c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f650bfff640 1 -- 172.21.15.72:0/2290178814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c19fd70 con 0x7f650c1029b0 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f650b7fe640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650c100ea0 0x7f650c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f650bfff640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 0x7f650c19f460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6504031a20 tx=0x7f6504003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f6508ff9640 1 -- 172.21.15.72:0/2290178814 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504004240 con 0x7f650c1029b0 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f6508ff9640 1 -- 172.21.15.72:0/2290178814 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6504003e10 con 0x7f650c1029b0 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c19ffa0 con 0x7f650c1029b0 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f6508ff9640 1 -- 172.21.15.72:0/2290178814 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040305c0 con 0x7f650c1029b0 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.894+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c1a0460 con 0x7f650c1029b0 2024-04-25T13:12:54.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.895+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650c108150 con 0x7f650c1029b0 2024-04-25T13:12:54.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.896+0000 7f6508ff9640 1 -- 172.21.15.72:0/2290178814 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f650402f660 con 0x7f650c1029b0 2024-04-25T13:12:54.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.896+0000 7f6508ff9640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f64f40a3bb0 0x7f64f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:54.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.897+0000 7f650b7fe640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f64f40a3bb0 0x7f64f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:54.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.897+0000 7f6508ff9640 1 -- 172.21.15.72:0/2290178814 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f65040f1d70 con 0x7f650c1029b0 2024-04-25T13:12:54.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.897+0000 7f650b7fe640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f64f40a3bb0 0x7f64f40a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00baa0 tx=0x7f64fc005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:54.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:54.901+0000 7f6508ff9640 1 -- 172.21.15.72:0/2290178814 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65040be740 con 0x7f650c1029b0 2024-04-25T13:12:55.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.118+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f650c062bf0 con 0x7f650c1029b0 2024-04-25T13:12:55.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.119+0000 7f6508ff9640 1 -- 172.21.15.72:0/2290178814 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11881 (secure 0 0 0) 0x7f65040be040 con 0x7f650c1029b0 2024-04-25T13:12:55.119 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:55.120 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:11:04.601639+0000","modified":"2024-04-25T13:11:09.292132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24493,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6839/1444196308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1444196308},{"type":"v1","addr":"172.21.15.164:6839","nonce":1444196308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24478,24493,24509]},"id":1}]} 2024-04-25T13:12:55.120 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 20 2024-04-25T13:12:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.121+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f64f40a3bb0 msgr2=0x7f64f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f64f40a3bb0 0x7f64f40a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00baa0 tx=0x7f64fc005d90 comp rx=0 tx=0).stop 2024-04-25T13:12:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 msgr2=0x7f650c19f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 0x7f650c19f460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6504031a20 tx=0x7f6504003c20 comp rx=0 tx=0).stop 2024-04-25T13:12:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 shutdown_connections 2024-04-25T13:12:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f64f40a3bb0 0x7f64f40a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f650c1029b0 0x7f650c19f460 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f650c101ff0 0x7f650c19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 --2- 172.21.15.72:0/2290178814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650c100ea0 0x7f650c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 >> 172.21.15.72:0/2290178814 conn(0x7f650c0fc430 msgr2=0x7f650c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 shutdown_connections 2024-04-25T13:12:55.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.122+0000 7f6511dad640 1 -- 172.21.15.72:0/2290178814 wait complete. 2024-04-25T13:12:55.134 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:12:55.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 -- 172.21.15.72:0/2134737313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c105360 msgr2=0x7fad7c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 --2- 172.21.15.72:0/2134737313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c105360 0x7fad7c107750 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fad78009900 tx=0x7fad7802a410 comp rx=0 tx=0).stop 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 -- 172.21.15.72:0/2134737313 shutdown_connections 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 --2- 172.21.15.72:0/2134737313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad7c107c90 0x7fad7c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 --2- 172.21.15.72:0/2134737313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c105360 0x7fad7c107750 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 --2- 172.21.15.72:0/2134737313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad7c102a30 0x7fad7c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 -- 172.21.15.72:0/2134737313 >> 172.21.15.72:0/2134737313 conn(0x7fad7c0fc440 msgr2=0x7fad7c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 -- 172.21.15.72:0/2134737313 shutdown_connections 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 -- 172.21.15.72:0/2134737313 wait complete. 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.234+0000 7fad83b56640 1 Processor -- start 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad83b56640 1 -- start start 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad83b56640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad7c102a30 0x7fad7c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad83b56640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad7c105360 0x7fad7c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad83b56640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c107c90 0x7fad7c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad83b56640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10db30 con 0x7fad7c107c90 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad83b56640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10d9b0 con 0x7fad7c102a30 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad83b56640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10dcb0 con 0x7fad7c105360 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad820f3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c107c90 0x7fad7c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad820f3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c107c90 0x7fad7c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34030/0 (socket says 172.21.15.72:34030) 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad820f3640 1 -- 172.21.15.72:0/3607587535 learned_addr learned my addr 172.21.15.72:0/3607587535 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad810f1640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad7c105360 0x7fad7c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad818f2640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad7c102a30 0x7fad7c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad820f3640 1 -- 172.21.15.72:0/3607587535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad7c105360 msgr2=0x7fad7c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad820f3640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad7c105360 0x7fad7c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad820f3640 1 -- 172.21.15.72:0/3607587535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad7c102a30 msgr2=0x7fad7c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad820f3640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad7c102a30 0x7fad7c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad820f3640 1 -- 172.21.15.72:0/3607587535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c19fea0 con 0x7fad7c107c90 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad810f1640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad7c105360 0x7fad7c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad818f2640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad7c102a30 0x7fad7c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad820f3640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c107c90 0x7fad7c19f830 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fad7400b970 tx=0x7fad7400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad72ffd640 1 -- 172.21.15.72:0/3607587535 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74019070 con 0x7fad7c107c90 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c1a0190 con 0x7fad7c107c90 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad72ffd640 1 -- 172.21.15.72:0/3607587535 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad74014440 con 0x7fad7c107c90 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad72ffd640 1 -- 172.21.15.72:0/3607587535 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad740075d0 con 0x7fad7c107c90 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.235+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c1a06e0 con 0x7fad7c107c90 2024-04-25T13:12:55.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.236+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7c0fdf30 con 0x7fad7c107c90 2024-04-25T13:12:55.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.238+0000 7fad72ffd640 1 -- 172.21.15.72:0/3607587535 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fad7401c070 con 0x7fad7c107c90 2024-04-25T13:12:55.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.238+0000 7fad72ffd640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad680a3bb0 0x7fad680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.238+0000 7fad818f2640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad680a3bb0 0x7fad680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.239+0000 7fad72ffd640 1 -- 172.21.15.72:0/3607587535 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fad740cffd0 con 0x7fad7c107c90 2024-04-25T13:12:55.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.239+0000 7fad818f2640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad680a3bb0 0x7fad680a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009b40 tx=0x7fad6c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.242+0000 7fad72ffd640 1 -- 172.21.15.72:0/3607587535 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad7409c940 con 0x7fad7c107c90 2024-04-25T13:12:55.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.422+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad7c02d5c0 con 0x7fad7c107c90 2024-04-25T13:12:55.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.423+0000 7fad72ffd640 1 -- 172.21.15.72:0/3607587535 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22740 (secure 0 0 0) 0x7fad7409c240 con 0x7fad7c107c90 2024-04-25T13:12:55.424 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:12:55.424 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:12:08.140624+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad680a3bb0 msgr2=0x7fad680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad680a3bb0 0x7fad680a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009b40 tx=0x7fad6c005e50 comp rx=0 tx=0).stop 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c107c90 msgr2=0x7fad7c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c107c90 0x7fad7c19f830 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fad7400b970 tx=0x7fad7400be30 comp rx=0 tx=0).stop 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 shutdown_connections 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fad680a3bb0 0x7fad680a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad7c107c90 0x7fad7c19f830 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad7c105360 0x7fad7c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 --2- 172.21.15.72:0/3607587535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad7c102a30 0x7fad7c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 >> 172.21.15.72:0/3607587535 conn(0x7fad7c0fc440 msgr2=0x7fad7c1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.425+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 shutdown_connections 2024-04-25T13:12:55.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.426+0000 7fad83b56640 1 -- 172.21.15.72:0/3607587535 wait complete. 2024-04-25T13:12:55.436 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:55 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2604842714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:55.436 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:55 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2453125863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:55.436 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:55 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2290178814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:55.438 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:11:04.601639+0000', 'modified': '2024-04-25T13:11:09.292132+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24493, 'mds_2': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/3356589609', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 3356589609}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 3356589609}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/1113925217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 1113925217}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 1113925217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6837/1335235896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 1335235896}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1335235896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.193:6837/1407296685', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1407296685}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1407296685}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.193:6839/2585421276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 2585421276}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 2585421276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6839/1444196308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 1444196308}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 1444196308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24478, 24493, 24509]} 2024-04-25T13:12:55.438 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T13:12:55.438 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T13:12:55.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.537+0000 7f164096f640 1 -- 172.21.15.72:0/1436647977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 msgr2=0x7f163c1015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.537+0000 7f164096f640 1 --2- 172.21.15.72:0/1436647977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 0x7f163c1015f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1630009900 tx=0x7f163002a410 comp rx=0 tx=0).stop 2024-04-25T13:12:55.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.537+0000 7f164096f640 1 -- 172.21.15.72:0/1436647977 shutdown_connections 2024-04-25T13:12:55.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.537+0000 7f164096f640 1 --2- 172.21.15.72:0/1436647977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f163c101b30 0x7f163c1093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.537+0000 7f164096f640 1 --2- 172.21.15.72:0/1436647977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 0x7f163c1015f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.537+0000 7f164096f640 1 --2- 172.21.15.72:0/1436647977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f163c102440 0x7f163c102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.537+0000 7f164096f640 1 -- 172.21.15.72:0/1436647977 >> 172.21.15.72:0/1436647977 conn(0x7f163c0fbbc0 msgr2=0x7f163c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.537+0000 7f164096f640 1 -- 172.21.15.72:0/1436647977 shutdown_connections 2024-04-25T13:12:55.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.537+0000 7f164096f640 1 -- 172.21.15.72:0/1436647977 wait complete. 2024-04-25T13:12:55.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f164096f640 1 Processor -- start 2024-04-25T13:12:55.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f164096f640 1 -- start start 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f164096f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 0x7f163c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f164096f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f163c101b30 0x7f163c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f164096f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f163c102440 0x7f163c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f164096f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10ca60 con 0x7f163c101170 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f164096f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10c8e0 con 0x7f163c101b30 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f164096f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10cbe0 con 0x7f163c102440 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f163a59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 0x7f163c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f163a59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 0x7f163c198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37058/0 (socket says 172.21.15.72:37058) 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f163a59c640 1 -- 172.21.15.72:0/1592315716 learned_addr learned my addr 172.21.15.72:0/1592315716 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f163a59c640 1 -- 172.21.15.72:0/1592315716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f163c102440 msgr2=0x7f163c19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f163a59c640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f163c102440 0x7f163c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f163a59c640 1 -- 172.21.15.72:0/1592315716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f163c101b30 msgr2=0x7f163c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f163a59c640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f163c101b30 0x7f163c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.538+0000 7f163a59c640 1 -- 172.21.15.72:0/1592315716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c19fea0 con 0x7f163c101170 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.539+0000 7f163a59c640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 0x7f163c198280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f162c00eaa0 tx=0x7f162c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.539+0000 7f162b7fe640 1 -- 172.21.15.72:0/1592315716 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c016070 con 0x7f163c101170 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.539+0000 7f162b7fe640 1 -- 172.21.15.72:0/1592315716 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f162c003e20 con 0x7f163c101170 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.539+0000 7f164096f640 1 -- 172.21.15.72:0/1592315716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163c1a0190 con 0x7f163c101170 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.539+0000 7f162b7fe640 1 -- 172.21.15.72:0/1592315716 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0116d0 con 0x7f163c101170 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.539+0000 7f164096f640 1 -- 172.21.15.72:0/1592315716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163c1a0710 con 0x7f163c101170 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.541+0000 7f162b7fe640 1 -- 172.21.15.72:0/1592315716 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f162c019070 con 0x7f163c101170 2024-04-25T13:12:55.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1643764740 1 Processor -- start 2024-04-25T13:12:55.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1643764740 1 -- start start 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1643764740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1643764740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 0x5635751b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1643764740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2b20 0x5635751b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1643764740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x563575111c50 con 0x5635751b2b20 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1643764740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5635750ae830 con 0x5635751b0f90 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1643764740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56357510dc10 con 0x5635751b2160 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1639d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2b20 0x5635751b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1639d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2b20 0x5635751b93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37074/0 (socket says 172.21.15.72:37074) 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1639d9b640 1 -- 172.21.15.72:0/1532400814 learned_addr learned my addr 172.21.15.72:0/1532400814 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f163ad9d640 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f162b7fe640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f16240a3bb0 0x7f16240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1639d9b640 1 -- 172.21.15.72:0/1532400814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 msgr2=0x5635751b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f163a59c640 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 0x5635751b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1639d9b640 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 0x5635751b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1639d9b640 1 -- 172.21.15.72:0/1532400814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 msgr2=0x5635751b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1639d9b640 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f1639d9b640 1 -- 172.21.15.72:0/1532400814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563574e242d0 con 0x5635751b2b20 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f163ad9d640 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.542+0000 7f163a59c640 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 0x5635751b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.543+0000 7f163ad9d640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f16240a3bb0 0x7f16240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.543+0000 7f162b7fe640 1 -- 172.21.15.72:0/1592315716 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f162c0d0590 con 0x7f163c101170 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.543+0000 7f1639d9b640 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2b20 0x5635751b93b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1630031620 tx=0x7f1630031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a73349719626db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.543+0000 7f1628ff9640 1 -- 172.21.15.72:0/1532400814 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163003e070 con 0x5635751b2b20 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.543+0000 7f163ad9d640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f16240a3bb0 0x7f16240a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5635751b2060 tx=0x7f1634008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.543+0000 7f1628ff9640 1 -- 172.21.15.72:0/1532400814 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1630004050 con 0x5635751b2b20 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.543+0000 7f1628ff9640 1 -- 172.21.15.72:0/1532400814 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630035040 con 0x5635751b2b20 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.543+0000 7f1643764740 1 -- 172.21.15.72:0/1532400814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2b20 msgr2=0x5635751b93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2b20 0x5635751b93b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1630031620 tx=0x7f1630031650 comp rx=0 tx=0).stop 2024-04-25T13:12:55.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 -- 172.21.15.72:0/1532400814 shutdown_connections 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2b20 0x5635751b93b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 0x5635751b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 --2- 172.21.15.72:0/1532400814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 -- 172.21.15.72:0/1532400814 >> 172.21.15.72:0/1532400814 conn(0x5635751a8070 msgr2=0x5635751a8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 -- 172.21.15.72:0/1532400814 shutdown_connections 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 -- 172.21.15.72:0/1532400814 wait complete. 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 -- start start 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 Processor -- start 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751bb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751bb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 0x5635751c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5635751b9ad0 con 0x5635751b2160 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5635750ac470 con 0x5635751b0f90 2024-04-25T13:12:55.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5635751b9c50 con 0x5635751b2b20 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1643764740 10 client.0 _ll_register_callbacks cb 0x5635750d1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1639d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1639d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751bb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37088/0 (socket says 172.21.15.72:37088) 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f1639d9b640 1 -- 172.21.15.72:0/766577116 learned_addr learned my addr 172.21.15.72:0/766577116 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f163a59c640 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.544+0000 7f163ad9d640 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 0x5635751c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1639d9b640 1 -- 172.21.15.72:0/766577116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 msgr2=0x5635751c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1639d9b640 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 0x5635751c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1639d9b640 1 -- 172.21.15.72:0/766577116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 msgr2=0x5635751bb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1639d9b640 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751bb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1639d9b640 1 -- 172.21.15.72:0/766577116 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563574eee6b0 con 0x5635751b2160 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1639d9b640 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751bb890 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1630005ab0 tx=0x7f1630031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6685ba4013856f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1621ffb640 1 -- 172.21.15.72:0/766577116 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163004a070 con 0x5635751b2160 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1621ffb640 1 -- 172.21.15.72:0/766577116 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1630031cd0 con 0x5635751b2160 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1643764740 1 -- 172.21.15.72:0/766577116 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1630004050 con 0x5635751b2160 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1621ffb640 1 -- 172.21.15.72:0/766577116 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1630056050 con 0x5635751b2160 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1643764740 10 client.15216 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1643764740 1 -- 172.21.15.72:0/766577116 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5635751c5bb0 con 0x5635751b2160 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1621ffb640 1 -- 172.21.15.72:0/766577116 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1630004050 con 0x5635751b2160 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1621ffb640 1 -- 172.21.15.72:0/766577116 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f16300435e0 con 0x5635751b2160 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1643764740 10 client.15216 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1643764740 10 client.15216 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1643764740 20 client.15216 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1643764740 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5635751c7d40 0x5635751c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1643764740 4 client.15216 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.545+0000 7f1643764740 1 -- 172.21.15.72:0/766577116 --> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56357511f670 con 0x5635751c7d40 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.546+0000 7f163a59c640 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5635751c7d40 0x5635751c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.546+0000 7f1621ffb640 10 client.15216 ms_handle_connect on v2:172.21.15.193:6836/1407296685 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.546+0000 7f163a59c640 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5635751c7d40 0x5635751c91b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fea77efb527e45c server_cookie=39487a8625a6fc70 in_seq=0 out_seq=0 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1621ffb640 1 -- 172.21.15.72:0/766577116 <== mds.2 v2:172.21.15.193:6836/1407296685 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56357511f670 con 0x5635751c7d40 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1621ffb640 10 client.15216 handle_command_reply: tid=0 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 client.15216 shutdown 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 -- 172.21.15.72:0/766577116 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 msgr2=0x5635751bb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751bb890 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1630005ab0 tx=0x7f1630031960 comp rx=0 tx=0).stop 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 -- 172.21.15.72:0/766577116 shutdown_connections 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 0x5635751c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751bb890 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x5635751c7d40 0x5635751c91b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 --2- 172.21.15.72:0/766577116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751bb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1621ffb640 0 client.15216 ms_handle_reset on v2:172.21.15.193:6836/1407296685 2024-04-25T13:12:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 -- 172.21.15.72:0/766577116 >> 172.21.15.72:0/766577116 conn(0x5635751a8070 msgr2=0x5635751a71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 -- 172.21.15.72:0/766577116 shutdown_connections 2024-04-25T13:12:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 1 -- 172.21.15.72:0/766577116 wait complete. 2024-04-25T13:12:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.547+0000 7f1643764740 20 client.15216 trim_cache size 0 max 16384 2024-04-25T13:12:55.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:55 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2604842714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:55.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:55 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2453125863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:55.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:55 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2290178814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:55.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f1643764740 1 Processor -- start 2024-04-25T13:12:55.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f1643764740 1 -- start start 2024-04-25T13:12:55.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f1643764740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b0f90 0x5635751b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f1643764740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 0x5635751b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f1643764740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b2b20 0x5635751c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f1643764740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5635751c74f0 con 0x5635751b0f90 2024-04-25T13:12:55.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f1643764740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5635751c7bc0 con 0x5635751b2b20 2024-04-25T13:12:55.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f1643764740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5635751c8290 con 0x5635751b2160 2024-04-25T13:12:55.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f163a59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b0f90 0x5635751b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f163a59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b0f90 0x5635751b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37102/0 (socket says 172.21.15.72:37102) 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f163a59c640 1 -- 172.21.15.72:0/4079467696 learned_addr learned my addr 172.21.15.72:0/4079467696 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f163a59c640 1 -- 172.21.15.72:0/4079467696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 msgr2=0x5635751b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f163a59c640 1 --2- 172.21.15.72:0/4079467696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 0x5635751b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f163a59c640 1 -- 172.21.15.72:0/4079467696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b2b20 msgr2=0x5635751c2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f163a59c640 1 --2- 172.21.15.72:0/4079467696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b2b20 0x5635751c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f163a59c640 1 -- 172.21.15.72:0/4079467696 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635751a7ea0 con 0x5635751b0f90 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.565+0000 7f163a59c640 1 --2- 172.21.15.72:0/4079467696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b0f90 0x5635751b5c00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f162c004530 tx=0x7f162c06f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fe363a313c5718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1623fff640 1 -- 172.21.15.72:0/4079467696 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c020040 con 0x5635751b0f90 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1623fff640 1 -- 172.21.15.72:0/4079467696 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f162c09c860 con 0x5635751b0f90 2024-04-25T13:12:55.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1623fff640 1 -- 172.21.15.72:0/4079467696 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c09ab60 con 0x5635751b0f90 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 -- 172.21.15.72:0/4079467696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b0f90 msgr2=0x5635751b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 --2- 172.21.15.72:0/4079467696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b0f90 0x5635751b5c00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f162c004530 tx=0x7f162c06f3a0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 -- 172.21.15.72:0/4079467696 shutdown_connections 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 --2- 172.21.15.72:0/4079467696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b2b20 0x5635751c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 --2- 172.21.15.72:0/4079467696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2160 0x5635751b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 --2- 172.21.15.72:0/4079467696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b0f90 0x5635751b5c00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 -- 172.21.15.72:0/4079467696 >> 172.21.15.72:0/4079467696 conn(0x5635751a8070 msgr2=0x5635751a8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 -- 172.21.15.72:0/4079467696 shutdown_connections 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 -- 172.21.15.72:0/4079467696 wait complete. 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.566+0000 7f1643764740 1 -- start start 2024-04-25T13:12:55.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 1 Processor -- start 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751bdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751be430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 0x5635751c1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5635751c9030 con 0x5635751b2160 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5635751c8960 con 0x5635751b0f90 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5635751c9700 con 0x5635751b2b20 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 10 client.0 _ll_register_callbacks cb 0x7f1630005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1639d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751be430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1639d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751be430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37116/0 (socket says 172.21.15.72:37116) 2024-04-25T13:12:55.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1639d9b640 1 -- 172.21.15.72:0/1569004616 learned_addr learned my addr 172.21.15.72:0/1569004616 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f163ad9d640 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 0x5635751c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1639d9b640 1 -- 172.21.15.72:0/1569004616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 msgr2=0x5635751c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1639d9b640 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 0x5635751c1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1639d9b640 1 -- 172.21.15.72:0/1569004616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 msgr2=0x5635751bdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1639d9b640 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751bdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1639d9b640 1 -- 172.21.15.72:0/1569004616 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635751c1a40 con 0x5635751b2160 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1639d9b640 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751be430 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f163002a930 tx=0x7f1630043700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1f31394d3711c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1621ffb640 1 -- 172.21.15.72:0/1569004616 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163004a070 con 0x5635751b2160 2024-04-25T13:12:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1621ffb640 1 -- 172.21.15.72:0/1569004616 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1630031c80 con 0x5635751b2160 2024-04-25T13:12:55.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 1 -- 172.21.15.72:0/1569004616 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f162c09c860 con 0x5635751b2160 2024-04-25T13:12:55.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1621ffb640 1 -- 172.21.15.72:0/1569004616 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1630062050 con 0x5635751b2160 2024-04-25T13:12:55.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 10 client.15222 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:55.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.567+0000 7f1643764740 1 -- 172.21.15.72:0/1569004616 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56357522dd20 con 0x5635751b2160 2024-04-25T13:12:55.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f1621ffb640 1 -- 172.21.15.72:0/1569004616 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f162c09c860 con 0x5635751b2160 2024-04-25T13:12:55.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f1621ffb640 1 -- 172.21.15.72:0/1569004616 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f163005bc90 con 0x5635751b2160 2024-04-25T13:12:55.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f1643764740 10 client.15222 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:55.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f1643764740 10 client.15222 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:12:55.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f1643764740 20 client.15222 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f1643764740 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x563575232890 0x5635752315a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f1643764740 4 client.15222 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f1643764740 1 -- 172.21.15.72:0/1569004616 --> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] -- command(tid 0: {"prefix": "status"}) -- 0x5635751194e0 con 0x563575232890 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f163a59c640 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x563575232890 0x5635752315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f163a59c640 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x563575232890 0x5635752315a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e409576ab272806 server_cookie=12e7f5baa9cbcc74 in_seq=0 out_seq=0 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.568+0000 7f1621ffb640 10 client.15222 ms_handle_connect on v2:172.21.15.193:6836/1407296685 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1621ffb640 1 -- 172.21.15.72:0/1569004616 <== mds.2 v2:172.21.15.193:6836/1407296685 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5635751194e0 con 0x563575232890 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1621ffb640 10 client.15222 handle_command_reply: tid=0 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 client.15222 shutdown 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 -- 172.21.15.72:0/1569004616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 msgr2=0x5635751be430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751be430 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f163002a930 tx=0x7f1630043700 comp rx=0 tx=0).stop 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 -- 172.21.15.72:0/1569004616 shutdown_connections 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635751b2b20 0x5635751c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5635751b2160 0x5635751be430 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.193:6836/1407296685,v1:172.21.15.193:6837/1407296685] conn(0x563575232890 0x5635752315a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 --2- 172.21.15.72:0/1569004616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635751b0f90 0x5635751bdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1621ffb640 0 client.15222 ms_handle_reset on v2:172.21.15.193:6836/1407296685 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 -- 172.21.15.72:0/1569004616 >> 172.21.15.72:0/1569004616 conn(0x5635751a8070 msgr2=0x5635751a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 -- 172.21.15.72:0/1569004616 shutdown_connections 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 1 -- 172.21.15.72:0/1569004616 wait complete. 2024-04-25T13:12:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f1643764740 20 client.15222 trim_cache size 0 max 16384 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 2, 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24478, 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 107.28556765, 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 20, 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 90, 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 130.34863743599999 2024-04-25T13:12:55.579 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 -- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f16240a3bb0 msgr2=0x7f16240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f16240a3bb0 0x7f16240a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5635751b2060 tx=0x7f1634008040 comp rx=0 tx=0).stop 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 -- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 msgr2=0x7f163c198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 0x7f163c198280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f162c00eaa0 tx=0x7f162c00ef60 comp rx=0 tx=0).stop 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 -- 172.21.15.72:0/1592315716 shutdown_connections 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f16240a3bb0 0x7f16240a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f163c102440 0x7f163c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f163c101b30 0x7f163c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 --2- 172.21.15.72:0/1592315716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f163c101170 0x7f163c198280 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 -- 172.21.15.72:0/1592315716 >> 172.21.15.72:0/1592315716 conn(0x7f163c0fbbc0 msgr2=0x7f163c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 -- 172.21.15.72:0/1592315716 shutdown_connections 2024-04-25T13:12:55.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.569+0000 7f164096f640 1 -- 172.21.15.72:0/1592315716 wait complete. 2024-04-25T13:12:55.582 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:12:55.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 -- 172.21.15.72:0/1697136337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102a40 msgr2=0x7f9c04109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 --2- 172.21.15.72:0/1697136337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102a40 0x7f9c04109360 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009ca0 tx=0x7f9bfc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 -- 172.21.15.72:0/1697136337 shutdown_connections 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 --2- 172.21.15.72:0/1697136337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102a40 0x7f9c04109360 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 --2- 172.21.15.72:0/1697136337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04102080 0x7f9c04102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 --2- 172.21.15.72:0/1697136337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c04100f30 0x7f9c04101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 -- 172.21.15.72:0/1697136337 >> 172.21.15.72:0/1697136337 conn(0x7f9c040fc4a0 msgr2=0x7f9c040fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 -- 172.21.15.72:0/1697136337 shutdown_connections 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 -- 172.21.15.72:0/1697136337 wait complete. 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 Processor -- start 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 -- start start 2024-04-25T13:12:55.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c04100f30 0x7f9c041981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102080 0x7f9c04198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04102a40 0x7f9c0419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0410ca90 con 0x7f9c04102080 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0410c910 con 0x7f9c04100f30 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.682+0000 7f9c0c205640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0410cc10 con 0x7f9c04102a40 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c097a0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102080 0x7f9c04198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c097a0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102080 0x7f9c04198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37130/0 (socket says 172.21.15.72:37130) 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c097a0640 1 -- 172.21.15.72:0/4138921671 learned_addr learned my addr 172.21.15.72:0/4138921671 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04102a40 0x7f9c0419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c04100f30 0x7f9c041981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c097a0640 1 -- 172.21.15.72:0/4138921671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04102a40 msgr2=0x7f9c0419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c097a0640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04102a40 0x7f9c0419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c097a0640 1 -- 172.21.15.72:0/4138921671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c04100f30 msgr2=0x7f9c041981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c097a0640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c04100f30 0x7f9c041981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c097a0640 1 -- 172.21.15.72:0/4138921671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0419fe00 con 0x7f9c04102080 2024-04-25T13:12:55.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04102a40 0x7f9c0419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:55.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c097a0640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102080 0x7f9c04198720 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000d950 tx=0x7f9c0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9bfaffd640 1 -- 172.21.15.72:0/4138921671 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00017070 con 0x7f9c04102080 2024-04-25T13:12:55.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9bfaffd640 1 -- 172.21.15.72:0/4138921671 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c00009c30 con 0x7f9c04102080 2024-04-25T13:12:55.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c0c205640 1 -- 172.21.15.72:0/4138921671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c041a00f0 con 0x7f9c04102080 2024-04-25T13:12:55.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9bfaffd640 1 -- 172.21.15.72:0/4138921671 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000f620 con 0x7f9c04102080 2024-04-25T13:12:55.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.683+0000 7f9c0c205640 1 -- 172.21.15.72:0/4138921671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c041a05f0 con 0x7f9c04102080 2024-04-25T13:12:55.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.685+0000 7f9bfaffd640 1 -- 172.21.15.72:0/4138921671 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9c0001a070 con 0x7f9c04102080 2024-04-25T13:12:55.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.685+0000 7f9c0eff6740 1 Processor -- start 2024-04-25T13:12:55.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0eff6740 1 -- start start 2024-04-25T13:12:55.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd299fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 0x55bd299fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 0x55bd29a04fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bd29961c50 con 0x55bd299fcb80 2024-04-25T13:12:55.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bd298fe830 con 0x55bd299fe710 2024-04-25T13:12:55.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd2995dc10 con 0x55bd299fdd50 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9bfaffd640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f9bf00a3bb0 0x7f9bf00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0a7a2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd299fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0a7a2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd299fcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37134/0 (socket says 172.21.15.72:37134) 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0a7a2640 1 -- 172.21.15.72:0/3606252625 learned_addr learned my addr 172.21.15.72:0/3606252625 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f9bf00a3bb0 0x7f9bf00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c097a0640 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 0x55bd29a04fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 0x55bd299fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9bfaffd640 1 -- 172.21.15.72:0/4138921671 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f9c000cfeb0 con 0x7f9c04102080 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0a7a2640 1 -- 172.21.15.72:0/3606252625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 msgr2=0x55bd299fe1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 0x55bd299fe1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0a7a2640 1 -- 172.21.15.72:0/3606252625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 msgr2=0x55bd29a04fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 0x55bd29a04fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c0a7a2640 1 -- 172.21.15.72:0/3606252625 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd296742d0 con 0x55bd299fcb80 2024-04-25T13:12:55.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.686+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 0x55bd299fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd299fcfa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009430 tx=0x7f9bfc031600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fd8f002579535a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f9bf00a3bb0 0x7f9bf00a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc005e30 tx=0x7f9bfc043000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9beffff640 1 -- 172.21.15.72:0/3606252625 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc04e070 con 0x55bd299fcb80 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9beffff640 1 -- 172.21.15.72:0/3606252625 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc036040 con 0x55bd299fcb80 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9beffff640 1 -- 172.21.15.72:0/3606252625 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc02f8c0 con 0x55bd299fcb80 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9c0eff6740 1 -- 172.21.15.72:0/3606252625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 msgr2=0x55bd299fcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd299fcfa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009430 tx=0x7f9bfc031600 comp rx=0 tx=0).stop 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9c0eff6740 1 -- 172.21.15.72:0/3606252625 shutdown_connections 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 0x55bd29a04fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 0x55bd299fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/3606252625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd299fcfa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.687+0000 7f9c0eff6740 1 -- 172.21.15.72:0/3606252625 >> 172.21.15.72:0/3606252625 conn(0x55bd299f8070 msgr2=0x55bd299f8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 -- 172.21.15.72:0/3606252625 shutdown_connections 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 -- 172.21.15.72:0/3606252625 wait complete. 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 -- start start 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 Processor -- start 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a07060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fdd50 0x55bd29a075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 0x55bd29a0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bd29a056c0 con 0x55bd299fcb80 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bd298fc470 con 0x55bd299fdd50 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd29a05840 con 0x55bd299fe710 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c0eff6740 10 client.0 _ll_register_callbacks cb 0x55bd29921f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c09fa1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a07060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c09fa1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a07060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37136/0 (socket says 172.21.15.72:37136) 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c09fa1640 1 -- 172.21.15.72:0/2402172416 learned_addr learned my addr 172.21.15.72:0/2402172416 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.688+0000 7f9c097a0640 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fdd50 0x55bd29a075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 0x55bd29a0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c09fa1640 1 -- 172.21.15.72:0/2402172416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 msgr2=0x55bd29a0e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 0x55bd29a0e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c09fa1640 1 -- 172.21.15.72:0/2402172416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fdd50 msgr2=0x55bd29a075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fdd50 0x55bd29a075a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c09fa1640 1 -- 172.21.15.72:0/2402172416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd2973e6b0 con 0x55bd299fcb80 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 0x55bd29a0e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a07060 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400df20 tx=0x7f9bf4011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7db62b648fae79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9bed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9bed7fa640 1 -- 172.21.15.72:0/2402172416 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4004230 con 0x55bd299fcb80 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9bed7fa640 1 -- 172.21.15.72:0/2402172416 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f9bf40043d0 con 0x55bd299fcb80 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2402172416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bfc036040 con 0x55bd299fcb80 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9bed7fa640 1 -- 172.21.15.72:0/2402172416 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9bf40035f0 con 0x55bd299fcb80 2024-04-25T13:12:55.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c0eff6740 10 client.15234 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2402172416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bd29a11a20 con 0x55bd299fcb80 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9bed7fa640 1 -- 172.21.15.72:0/2402172416 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc036040 con 0x55bd299fcb80 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9bed7fa640 1 -- 172.21.15.72:0/2402172416 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f9bf401b970 con 0x55bd299fcb80 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.689+0000 7f9c0eff6740 10 client.15234 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.690+0000 7f9c0eff6740 10 client.15234 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.690+0000 7f9c0eff6740 20 client.15234 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.690+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x55bd29a14090 0x55bd29a15500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.690+0000 7f9c0eff6740 4 client.15234 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.690+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2402172416 --> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd2996f670 con 0x55bd29a14090 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.690+0000 7f9c097a0640 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x55bd29a14090 0x55bd29a15500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.690+0000 7f9bed7fa640 10 client.15234 ms_handle_connect on v2:172.21.15.193:6838/2585421276 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.690+0000 7f9c097a0640 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x55bd29a14090 0x55bd29a15500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d5488c17c6b4ea4 server_cookie=d420739bec8da92c in_seq=0 out_seq=0 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9bed7fa640 1 -- 172.21.15.72:0/2402172416 <== mds.1 v2:172.21.15.193:6838/2585421276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd2996f670 con 0x55bd29a14090 2024-04-25T13:12:55.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9bed7fa640 10 client.15234 handle_command_reply: tid=0 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 client.15234 shutdown 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2402172416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 msgr2=0x55bd29a07060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a07060 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400df20 tx=0x7f9bf4011870 comp rx=0 tx=0).stop 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2402172416 shutdown_connections 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x55bd29a14090 0x55bd29a15500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 0x55bd29a0e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fdd50 0x55bd29a075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9bed7fa640 0 client.15234 ms_handle_reset on v2:172.21.15.193:6838/2585421276 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2402172416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a07060 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2402172416 >> 172.21.15.72:0/2402172416 conn(0x55bd299f8070 msgr2=0x55bd299f71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2402172416 shutdown_connections 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2402172416 wait complete. 2024-04-25T13:12:55.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.691+0000 7f9c0eff6740 20 client.15234 trim_cache size 0 max 16384 2024-04-25T13:12:55.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c0eff6740 1 Processor -- start 2024-04-25T13:12:55.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c0eff6740 1 -- start start 2024-04-25T13:12:55.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fcb80 0x55bd299fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fdd50 0x55bd29a0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 0x55bd299fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bd29a13840 con 0x55bd299fdd50 2024-04-25T13:12:55.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bd29a13f10 con 0x55bd299fcb80 2024-04-25T13:12:55.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd29a145e0 con 0x55bd299fe710 2024-04-25T13:12:55.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c097a0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fdd50 0x55bd29a0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c097a0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fdd50 0x55bd29a0e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37138/0 (socket says 172.21.15.72:37138) 2024-04-25T13:12:55.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.709+0000 7f9c097a0640 1 -- 172.21.15.72:0/2202505526 learned_addr learned my addr 172.21.15.72:0/2202505526 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/2202505526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fcb80 0x55bd299fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9c097a0640 1 -- 172.21.15.72:0/2202505526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 msgr2=0x55bd299fb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9c097a0640 1 --2- 172.21.15.72:0/2202505526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 0x55bd299fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9c097a0640 1 -- 172.21.15.72:0/2202505526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fcb80 msgr2=0x55bd299fafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9c097a0640 1 --2- 172.21.15.72:0/2202505526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fcb80 0x55bd299fafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9c097a0640 1 -- 172.21.15.72:0/2202505526 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd29a04bb0 con 0x55bd299fdd50 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/2202505526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fcb80 0x55bd299fafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9c097a0640 1 --2- 172.21.15.72:0/2202505526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fdd50 0x55bd29a0e570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009610 tx=0x7f9c00016500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7d3fa787c7cfe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9bef7fe640 1 -- 172.21.15.72:0/2202505526 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000cde60 con 0x55bd299fdd50 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9bef7fe640 1 -- 172.21.15.72:0/2202505526 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c000a0530 con 0x55bd299fdd50 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.710+0000 7f9bef7fe640 1 -- 172.21.15.72:0/2202505526 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00021040 con 0x55bd299fdd50 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2202505526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fdd50 msgr2=0x55bd29a0e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2202505526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fdd50 0x55bd29a0e570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009610 tx=0x7f9c00016500 comp rx=0 tx=0).stop 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2202505526 shutdown_connections 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2202505526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fe710 0x55bd299fb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2202505526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fdd50 0x55bd29a0e570 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/2202505526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fcb80 0x55bd299fafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2202505526 >> 172.21.15.72:0/2202505526 conn(0x55bd299f8070 msgr2=0x55bd299f30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2202505526 shutdown_connections 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 -- 172.21.15.72:0/2202505526 wait complete. 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 -- start start 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 Processor -- start 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a09c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 0x55bd29a0a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 0x55bd29a0d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bd29a15380 con 0x55bd299fcb80 2024-04-25T13:12:55.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bd29a14cb0 con 0x55bd299fe710 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd29a15a50 con 0x55bd299fdd50 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c0eff6740 10 client.0 _ll_register_callbacks cb 0x7f9bf400d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c09fa1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a09c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c09fa1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a09c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37150/0 (socket says 172.21.15.72:37150) 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c09fa1640 1 -- 172.21.15.72:0/736061117 learned_addr learned my addr 172.21.15.72:0/736061117 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.711+0000 7f9c097a0640 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 0x55bd29a0a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 0x55bd29a0d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c09fa1640 1 -- 172.21.15.72:0/736061117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 msgr2=0x55bd29a0a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 0x55bd29a0a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c09fa1640 1 -- 172.21.15.72:0/736061117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 msgr2=0x55bd29a0d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 0x55bd29a0d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c09fa1640 1 -- 172.21.15.72:0/736061117 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd29a0d7e0 con 0x55bd299fcb80 2024-04-25T13:12:55.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c0a7a2640 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 0x55bd29a0d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c09fa1640 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a09c90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4006010 tx=0x7f9bf4011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d012104b6392be3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9bed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9bed7fa640 1 -- 172.21.15.72:0/736061117 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf402b050 con 0x55bd299fcb80 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9bed7fa640 1 -- 172.21.15.72:0/736061117 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f9bf4012dc0 con 0x55bd299fcb80 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c0eff6740 1 -- 172.21.15.72:0/736061117 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c000a0530 con 0x55bd299fcb80 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9bed7fa640 1 -- 172.21.15.72:0/736061117 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9bf401b470 con 0x55bd299fcb80 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c0eff6740 10 client.15240 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c0eff6740 1 -- 172.21.15.72:0/736061117 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bd29a7a0b0 con 0x55bd299fcb80 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9bed7fa640 1 -- 172.21.15.72:0/736061117 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c000a0530 con 0x55bd299fcb80 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9bed7fa640 1 -- 172.21.15.72:0/736061117 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f9bf40309b0 con 0x55bd299fcb80 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.712+0000 7f9c0eff6740 10 client.15240 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:12:55.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 20 client.15240 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x55bd29a7ed20 0x55bd29a7da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 4 client.15240 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 -- 172.21.15.72:0/736061117 --> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] -- command(tid 0: {"prefix": "status"}) -- 0x55bd299694e0 con 0x55bd29a7ed20 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c097a0640 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x55bd29a7ed20 0x55bd29a7da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c097a0640 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x55bd29a7ed20 0x55bd29a7da30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0301bde8c2eb253 server_cookie=8591eb074774d993 in_seq=0 out_seq=0 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9bed7fa640 10 client.15240 ms_handle_connect on v2:172.21.15.193:6838/2585421276 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9bed7fa640 1 -- 172.21.15.72:0/736061117 <== mds.1 v2:172.21.15.193:6838/2585421276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bd299694e0 con 0x55bd29a7ed20 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9bed7fa640 10 client.15240 handle_command_reply: tid=0 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 client.15240 shutdown 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 -- 172.21.15.72:0/736061117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 msgr2=0x55bd29a09c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a09c90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4006010 tx=0x7f9bf4011df0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 -- 172.21.15.72:0/736061117 shutdown_connections 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.193:6838/2585421276,v1:172.21.15.193:6839/2585421276] conn(0x55bd29a7ed20 0x55bd29a7da30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd299fe710 0x55bd29a0d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd299fdd50 0x55bd29a0a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9bed7fa640 0 client.15240 ms_handle_reset on v2:172.21.15.193:6838/2585421276 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.713+0000 7f9c0eff6740 1 --2- 172.21.15.72:0/736061117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd299fcb80 0x55bd29a09c90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0eff6740 1 -- 172.21.15.72:0/736061117 >> 172.21.15.72:0/736061117 conn(0x55bd299f8070 msgr2=0x55bd299f6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0eff6740 1 -- 172.21.15.72:0/736061117 shutdown_connections 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0eff6740 1 -- 172.21.15.72:0/736061117 wait complete. 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0eff6740 20 client.15240 trim_cache size 0 max 16384 2024-04-25T13:12:55.723 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 1, 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24493, 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 108.45840527599999, 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 20, 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 90, 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 125.307455721 2024-04-25T13:12:55.724 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 -- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f9bf00a3bb0 msgr2=0x7f9bf00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f9bf00a3bb0 0x7f9bf00a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc005e30 tx=0x7f9bfc043000 comp rx=0 tx=0).stop 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 -- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102080 msgr2=0x7f9c04198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102080 0x7f9c04198720 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000d950 tx=0x7f9c0000de10 comp rx=0 tx=0).stop 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 -- 172.21.15.72:0/4138921671 shutdown_connections 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f9bf00a3bb0 0x7f9bf00a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04102a40 0x7f9c0419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c04102080 0x7f9c04198720 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 --2- 172.21.15.72:0/4138921671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c04100f30 0x7f9c041981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 -- 172.21.15.72:0/4138921671 >> 172.21.15.72:0/4138921671 conn(0x7f9c040fc4a0 msgr2=0x7f9c0406a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 -- 172.21.15.72:0/4138921671 shutdown_connections 2024-04-25T13:12:55.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.714+0000 7f9c0c205640 1 -- 172.21.15.72:0/4138921671 wait complete. 2024-04-25T13:12:55.727 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:12:55.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:55 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2604842714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:12:55.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:55 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2453125863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:12:55.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:55 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2290178814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:12:55.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 -- 172.21.15.72:0/2445754651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103180 msgr2=0x7f328c103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 --2- 172.21.15.72:0/2445754651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103180 0x7f328c103600 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f327c009900 tx=0x7f327c02a410 comp rx=0 tx=0).stop 2024-04-25T13:12:55.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 -- 172.21.15.72:0/2445754651 shutdown_connections 2024-04-25T13:12:55.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 --2- 172.21.15.72:0/2445754651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f328c103b40 0x7f328c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 --2- 172.21.15.72:0/2445754651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103180 0x7f328c103600 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 --2- 172.21.15.72:0/2445754651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f328c101f90 0x7f328c1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 -- 172.21.15.72:0/2445754651 >> 172.21.15.72:0/2445754651 conn(0x7f328c0fd500 msgr2=0x7f328c0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 -- 172.21.15.72:0/2445754651 shutdown_connections 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 -- 172.21.15.72:0/2445754651 wait complete. 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 Processor -- start 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 -- start start 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f328c101f90 0x7f328c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f328c103180 0x7f328c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103b40 0x7f328c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f328c10db30 con 0x7f328c103b40 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f328c10d9b0 con 0x7f328c101f90 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.825+0000 7f329420b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f328c10dcb0 con 0x7f328c103180 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f32927a8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103b40 0x7f328c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f32927a8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103b40 0x7f328c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37176/0 (socket says 172.21.15.72:37176) 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f32927a8640 1 -- 172.21.15.72:0/3366492876 learned_addr learned my addr 172.21.15.72:0/3366492876 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f3291fa7640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f328c101f90 0x7f328c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f32927a8640 1 -- 172.21.15.72:0/3366492876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f328c103180 msgr2=0x7f328c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f32927a8640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f328c103180 0x7f328c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f32927a8640 1 -- 172.21.15.72:0/3366492876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f328c101f90 msgr2=0x7f328c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f32927a8640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f328c101f90 0x7f328c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f32927a8640 1 -- 172.21.15.72:0/3366492876 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c19fea0 con 0x7f328c103b40 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f32927a8640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103b40 0x7f328c19f830 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f328800d950 tx=0x7f328800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f3282ffd640 1 -- 172.21.15.72:0/3366492876 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288017030 con 0x7f328c103b40 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f3282ffd640 1 -- 172.21.15.72:0/3366492876 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3288004020 con 0x7f328c103b40 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f329420b640 1 -- 172.21.15.72:0/3366492876 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328c1a0190 con 0x7f328c103b40 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f3282ffd640 1 -- 172.21.15.72:0/3366492876 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288003b50 con 0x7f328c103b40 2024-04-25T13:12:55.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.826+0000 7f329420b640 1 -- 172.21.15.72:0/3366492876 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328c1a0690 con 0x7f328c103b40 2024-04-25T13:12:55.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.828+0000 7f3282ffd640 1 -- 172.21.15.72:0/3366492876 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f328801a070 con 0x7f328c103b40 2024-04-25T13:12:55.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f3296ff4740 1 Processor -- start 2024-04-25T13:12:55.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f3296ff4740 1 -- start start 2024-04-25T13:12:55.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c97705fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c977071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c9770dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f3296ff4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557c9766ac50 con 0x557c97705b80 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f3296ff4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c97607830 con 0x557c97707710 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f3296ff4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557c97666c10 con 0x557c97706d50 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f32917a6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c97705fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f32917a6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c97705fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37188/0 (socket says 172.21.15.72:37188) 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f3282ffd640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f32780a3bb0 0x7f32780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f32917a6640 1 -- 172.21.15.72:0/1643994150 learned_addr learned my addr 172.21.15.72:0/1643994150 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.829+0000 7f32927a8640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f32780a3bb0 0x7f32780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f3291fa7640 1 --2- 172.21.15.72:0/1643994150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c977071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f32917a6640 1 -- 172.21.15.72:0/1643994150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 msgr2=0x557c977071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f32917a6640 1 --2- 172.21.15.72:0/1643994150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c977071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f32917a6640 1 -- 172.21.15.72:0/1643994150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 msgr2=0x557c9770dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f32917a6640 1 --2- 172.21.15.72:0/1643994150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c9770dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f32917a6640 1 -- 172.21.15.72:0/1643994150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c9737d2d0 con 0x557c97705b80 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f3282ffd640 1 -- 172.21.15.72:0/3366492876 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f32880cfcb0 con 0x7f328c103b40 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f32917a6640 1 --2- 172.21.15.72:0/1643994150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c97705fa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f327c031500 tx=0x7f327c032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdfdd0344d223a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f32927a8640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f32780a3bb0 0x7f32780a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3288098380 tx=0x7f328806d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f3277fff640 1 -- 172.21.15.72:0/1643994150 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c040020 con 0x557c97705b80 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f3277fff640 1 -- 172.21.15.72:0/1643994150 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c004050 con 0x557c97705b80 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.830+0000 7f3277fff640 1 -- 172.21.15.72:0/1643994150 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c035040 con 0x557c97705b80 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 -- 172.21.15.72:0/1643994150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 msgr2=0x557c97705fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 --2- 172.21.15.72:0/1643994150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c97705fa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f327c031500 tx=0x7f327c032bd0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 -- 172.21.15.72:0/1643994150 shutdown_connections 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 --2- 172.21.15.72:0/1643994150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c9770dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 --2- 172.21.15.72:0/1643994150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c977071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 --2- 172.21.15.72:0/1643994150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c97705fa0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 -- 172.21.15.72:0/1643994150 >> 172.21.15.72:0/1643994150 conn(0x557c97701070 msgr2=0x557c97701470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 -- 172.21.15.72:0/1643994150 shutdown_connections 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 -- 172.21.15.72:0/1643994150 wait complete. 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 -- start start 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 Processor -- start 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c9770ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c977104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c97717530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557c9770e6c0 con 0x557c97705b80 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c97605470 con 0x557c97707710 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.831+0000 7f3296ff4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557c9770e840 con 0x557c97706d50 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32917a6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c9770ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f3296ff4740 10 client.0 _ll_register_callbacks cb 0x557c9762af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32917a6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c9770ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37196/0 (socket says 172.21.15.72:37196) 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32917a6640 1 -- 172.21.15.72:0/3906327037 learned_addr learned my addr 172.21.15.72:0/3906327037 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f3291fa7640 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c977104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32917a6640 1 -- 172.21.15.72:0/3906327037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 msgr2=0x557c977104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32917a6640 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c977104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32917a6640 1 -- 172.21.15.72:0/3906327037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 msgr2=0x557c97717530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32917a6640 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c97717530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32917a6640 1 -- 172.21.15.72:0/3906327037 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c974476b0 con 0x557c97705b80 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32917a6640 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c9770ffb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f327c005c40 tx=0x7f327c0315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa781b683cc0e01c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32757fa640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32757fa640 1 -- 172.21.15.72:0/3906327037 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c04f020 con 0x557c97705b80 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32757fa640 1 -- 172.21.15.72:0/3906327037 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f327c0037f0 con 0x557c97705b80 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f3296ff4740 1 -- 172.21.15.72:0/3906327037 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327c004050 con 0x557c97705b80 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32757fa640 1 -- 172.21.15.72:0/3906327037 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f327c05a050 con 0x557c97705b80 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f3296ff4740 10 client.15252 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f3296ff4740 1 -- 172.21.15.72:0/3906327037 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557c9771a8a0 con 0x557c97705b80 2024-04-25T13:12:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32757fa640 1 -- 172.21.15.72:0/3906327037 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c004050 con 0x557c97705b80 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.832+0000 7f32757fa640 1 -- 172.21.15.72:0/3906327037 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f327c004d10 con 0x557c97705b80 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.833+0000 7f3296ff4740 10 client.15252 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.833+0000 7f3296ff4740 10 client.15252 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.833+0000 7f3296ff4740 20 client.15252 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.833+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x557c9771caa0 0x557c9771df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.833+0000 7f3296ff4740 4 client.15252 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.833+0000 7f3296ff4740 1 -- 172.21.15.72:0/3906327037 --> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c97678670 con 0x557c9771caa0 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.833+0000 7f3291fa7640 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x557c9771caa0 0x557c9771df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.833+0000 7f3291fa7640 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x557c9771caa0 0x557c9771df10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae901a62332a8693 server_cookie=2c926a914edbf394 in_seq=0 out_seq=0 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.833+0000 7f32757fa640 10 client.15252 ms_handle_connect on v2:172.21.15.164:6838/1444196308 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f32757fa640 1 -- 172.21.15.72:0/3906327037 <== mds.0 v2:172.21.15.164:6838/1444196308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c97678670 con 0x557c9771caa0 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f32757fa640 10 client.15252 handle_command_reply: tid=0 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 client.15252 shutdown 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 -- 172.21.15.72:0/3906327037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 msgr2=0x557c9770ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c9770ffb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f327c005c40 tx=0x7f327c0315a0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 -- 172.21.15.72:0/3906327037 shutdown_connections 2024-04-25T13:12:55.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x557c9771caa0 0x557c9771df10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c97717530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c977104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f32757fa640 0 client.15252 ms_handle_reset on v2:172.21.15.164:6838/1444196308 2024-04-25T13:12:55.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3906327037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97705b80 0x557c9770ffb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 -- 172.21.15.72:0/3906327037 >> 172.21.15.72:0/3906327037 conn(0x557c97701070 msgr2=0x557c977001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 -- 172.21.15.72:0/3906327037 shutdown_connections 2024-04-25T13:12:55.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.834+0000 7f3296ff4740 1 -- 172.21.15.72:0/3906327037 wait complete. 2024-04-25T13:12:55.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.835+0000 7f3296ff4740 20 client.15252 trim_cache size 0 max 16384 2024-04-25T13:12:55.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f3296ff4740 1 Processor -- start 2024-04-25T13:12:55.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f3296ff4740 1 -- start start 2024-04-25T13:12:55.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97705b80 0x557c97709f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c9771bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97707710 0x557c9770fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f3296ff4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557c9771c250 con 0x557c97707710 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f3296ff4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c9771c920 con 0x557c97705b80 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f3296ff4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557c9771cff0 con 0x557c97706d50 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f32917a6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97707710 0x557c9770fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f32917a6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97707710 0x557c9770fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37206/0 (socket says 172.21.15.72:37206) 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f32917a6640 1 -- 172.21.15.72:0/2892041801 learned_addr learned my addr 172.21.15.72:0/2892041801 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f32917a6640 1 -- 172.21.15.72:0/2892041801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 msgr2=0x557c9771bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f32917a6640 1 --2- 172.21.15.72:0/2892041801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c9771bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f32917a6640 1 -- 172.21.15.72:0/2892041801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97705b80 msgr2=0x557c97709f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f32917a6640 1 --2- 172.21.15.72:0/2892041801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97705b80 0x557c97709f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.852+0000 7f32917a6640 1 -- 172.21.15.72:0/2892041801 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c97705560 con 0x557c97707710 2024-04-25T13:12:55.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f32917a6640 1 --2- 172.21.15.72:0/2892041801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97707710 0x557c9770fbc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f327c005710 tx=0x7f327c03c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f77bbc25db2363f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f32777fe640 1 -- 172.21.15.72:0/2892041801 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c035070 con 0x557c97707710 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f32777fe640 1 -- 172.21.15.72:0/2892041801 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c04c040 con 0x557c97707710 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f32777fe640 1 -- 172.21.15.72:0/2892041801 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c04f040 con 0x557c97707710 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f3296ff4740 1 -- 172.21.15.72:0/2892041801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97707710 msgr2=0x557c9770fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f3296ff4740 1 --2- 172.21.15.72:0/2892041801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97707710 0x557c9770fbc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f327c005710 tx=0x7f327c03c890 comp rx=0 tx=0).stop 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f3296ff4740 1 -- 172.21.15.72:0/2892041801 shutdown_connections 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f3296ff4740 1 --2- 172.21.15.72:0/2892041801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97707710 0x557c9770fbc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f3296ff4740 1 --2- 172.21.15.72:0/2892041801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97706d50 0x557c9771bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f3296ff4740 1 --2- 172.21.15.72:0/2892041801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97705b80 0x557c97709f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f3296ff4740 1 -- 172.21.15.72:0/2892041801 >> 172.21.15.72:0/2892041801 conn(0x557c97701070 msgr2=0x557c976fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f3296ff4740 1 -- 172.21.15.72:0/2892041801 shutdown_connections 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.853+0000 7f3296ff4740 1 -- 172.21.15.72:0/2892041801 wait complete. 2024-04-25T13:12:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3296ff4740 1 -- start start 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3296ff4740 1 Processor -- start 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97705b80 0x557c97711ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97706d50 0x557c97712530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3296ff4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c97715550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3296ff4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557c9771dd90 con 0x557c97706d50 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3296ff4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c9771d6c0 con 0x557c97707710 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3296ff4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557c9771e460 con 0x557c97705b80 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3296ff4740 10 client.0 _ll_register_callbacks cb 0x7f327c032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3291fa7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97706d50 0x557c97712530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3291fa7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97706d50 0x557c97712530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37212/0 (socket says 172.21.15.72:37212) 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3291fa7640 1 -- 172.21.15.72:0/3868190558 learned_addr learned my addr 172.21.15.72:0/3868190558 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f32917a6640 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97705b80 0x557c97711ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f32917a6640 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c97715550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3291fa7640 1 -- 172.21.15.72:0/3868190558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97705b80 msgr2=0x557c97711ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3291fa7640 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97705b80 0x557c97711ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3291fa7640 1 -- 172.21.15.72:0/3868190558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 msgr2=0x557c97715550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3291fa7640 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c97715550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.854+0000 7f3291fa7640 1 -- 172.21.15.72:0/3868190558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c97715b40 con 0x557c97706d50 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3291fa7640 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97706d50 0x557c97712530 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3284009aa0 tx=0x7f328400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ac5a8fc22cf52e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f32757fa640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f32757fa640 1 -- 172.21.15.72:0/3868190558 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328401f070 con 0x557c97706d50 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f32757fa640 1 -- 172.21.15.72:0/3868190558 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f328401b5b0 con 0x557c97706d50 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3296ff4740 1 -- 172.21.15.72:0/3868190558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327c04c040 con 0x557c97706d50 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f32757fa640 1 -- 172.21.15.72:0/3868190558 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f328402a050 con 0x557c97706d50 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3296ff4740 10 client.15258 fetch_fsmap learned FSMap version 20 2024-04-25T13:12:55.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3296ff4740 1 -- 172.21.15.72:0/3868190558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557c97786e60 con 0x557c97706d50 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f32757fa640 1 -- 172.21.15.72:0/3868190558 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c04c040 con 0x557c97706d50 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f32757fa640 1 -- 172.21.15.72:0/3868190558 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f3284018410 con 0x557c97706d50 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3296ff4740 10 client.15258 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3296ff4740 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3296ff4740 20 client.15258 populate_metadata read hostname 'smithi072' 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x557c9778b9c0 0x557c9778a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3296ff4740 4 client.15258 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f3296ff4740 1 -- 172.21.15.72:0/3868190558 --> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] -- command(tid 0: {"prefix": "status"}) -- 0x557c976724e0 con 0x557c9778b9c0 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.855+0000 7f32917a6640 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x557c9778b9c0 0x557c9778a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f32757fa640 10 client.15258 ms_handle_connect on v2:172.21.15.164:6838/1444196308 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f32917a6640 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x557c9778b9c0 0x557c9778a6d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10407a7d3151b174 server_cookie=a81e099bf79c54bb in_seq=0 out_seq=0 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f32757fa640 1 -- 172.21.15.72:0/3868190558 <== mds.0 v2:172.21.15.164:6838/1444196308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557c976724e0 con 0x557c9778b9c0 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f32757fa640 10 client.15258 handle_command_reply: tid=0 2024-04-25T13:12:55.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 client.15258 shutdown 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 -- 172.21.15.72:0/3868190558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97706d50 msgr2=0x557c97712530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97706d50 0x557c97712530 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3284009aa0 tx=0x7f328400a890 comp rx=0 tx=0).stop 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 -- 172.21.15.72:0/3868190558 shutdown_connections 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.164:6838/1444196308,v1:172.21.15.164:6839/1444196308] conn(0x557c9778b9c0 0x557c9778a6d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c97707710 0x557c97715550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557c97706d50 0x557c97712530 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f32757fa640 0 client.15258 ms_handle_reset on v2:172.21.15.164:6838/1444196308 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 --2- 172.21.15.72:0/3868190558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c97705b80 0x557c97711ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 -- 172.21.15.72:0/3868190558 >> 172.21.15.72:0/3868190558 conn(0x557c97701070 msgr2=0x557c9770fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 -- 172.21.15.72:0/3868190558 shutdown_connections 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 1 -- 172.21.15.72:0/3868190558 wait complete. 2024-04-25T13:12:55.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.856+0000 7f3296ff4740 20 client.15258 trim_cache size 0 max 16384 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d8ca43ea-0303-11ef-bc93-c7b262605968", 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24509, 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 110.590373293, 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 20, 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 90, 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T13:12:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 117.167032557 2024-04-25T13:12:55.866 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:12:55.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 -- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f32780a3bb0 msgr2=0x7f32780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f32780a3bb0 0x7f32780a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3288098380 tx=0x7f328806d040 comp rx=0 tx=0).stop 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 -- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103b40 msgr2=0x7f328c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103b40 0x7f328c19f830 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f328800d950 tx=0x7f328800de10 comp rx=0 tx=0).stop 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 -- 172.21.15.72:0/3366492876 shutdown_connections 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f32780a3bb0 0x7f32780a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f328c103b40 0x7f328c19f830 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f328c103180 0x7f328c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 --2- 172.21.15.72:0/3366492876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f328c101f90 0x7f328c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 -- 172.21.15.72:0/3366492876 >> 172.21.15.72:0/3366492876 conn(0x7f328c0fd500 msgr2=0x7f328c106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 -- 172.21.15.72:0/3366492876 shutdown_connections 2024-04-25T13:12:55.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.857+0000 7f329420b640 1 -- 172.21.15.72:0/3366492876 wait complete. 2024-04-25T13:12:55.868 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T13:12:55.879 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T13:12:55.879 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T13:12:55.879 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T13:12:55.879 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T13:12:55.880 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:12:55.880 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:12:55.880 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T13:12:55.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.978+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/1571628507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 msgr2=0x7fc6d4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.978+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/1571628507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 0x7fc6d4101470 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009900 tx=0x7fc6c402a410 comp rx=0 tx=0).stop 2024-04-25T13:12:55.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.978+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/1571628507 shutdown_connections 2024-04-25T13:12:55.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.978+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/1571628507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d41019b0 0x7fc6d4106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.978+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/1571628507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 0x7fc6d4101470 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.978+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/1571628507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6d4100290 0x7fc6d41006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.978+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/1571628507 >> 172.21.15.72:0/1571628507 conn(0x7fc6d40fbbc0 msgr2=0x7fc6d40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/1571628507 shutdown_connections 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/1571628507 wait complete. 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 Processor -- start 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 -- start start 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6d4100290 0x7fc6d419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 0x7fc6d419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d41019b0 0x7fc6d41a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d4114120 con 0x7fc6d4100ff0 2024-04-25T13:12:55.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d4113fa0 con 0x7fc6d41019b0 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6dbf4a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d41142a0 con 0x7fc6d4100290 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6d94e5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 0x7fc6d419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6d94e5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 0x7fc6d419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37222/0 (socket says 172.21.15.72:37222) 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.979+0000 7fc6d94e5640 1 -- 172.21.15.72:0/3567599820 learned_addr learned my addr 172.21.15.72:0/3567599820 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6d9ce6640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6d4100290 0x7fc6d419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6da4e7640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d41019b0 0x7fc6d41a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6d94e5640 1 -- 172.21.15.72:0/3567599820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6d4100290 msgr2=0x7fc6d419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6d94e5640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6d4100290 0x7fc6d419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6d94e5640 1 -- 172.21.15.72:0/3567599820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d41019b0 msgr2=0x7fc6d41a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6d94e5640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d41019b0 0x7fc6d41a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6d94e5640 1 -- 172.21.15.72:0/3567599820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d41a63a0 con 0x7fc6d4100ff0 2024-04-25T13:12:55.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6da4e7640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d41019b0 0x7fc6d41a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:55.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6d9ce6640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6d4100290 0x7fc6d419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:12:55.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6d94e5640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 0x7fc6d419ed10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006010 tx=0x7fc6c4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6caffd640 1 -- 172.21.15.72:0/3567599820 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c403f030 con 0x7fc6d4100ff0 2024-04-25T13:12:55.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6caffd640 1 -- 172.21.15.72:0/3567599820 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c4003cb0 con 0x7fc6d4100ff0 2024-04-25T13:12:55.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d41a6630 con 0x7fc6d4100ff0 2024-04-25T13:12:55.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6caffd640 1 -- 172.21.15.72:0/3567599820 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4035070 con 0x7fc6d4100ff0 2024-04-25T13:12:55.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.980+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d41a6b20 con 0x7fc6d4100ff0 2024-04-25T13:12:55.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.981+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d4105060 con 0x7fc6d4100ff0 2024-04-25T13:12:55.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.982+0000 7fc6caffd640 1 -- 172.21.15.72:0/3567599820 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc6c4042030 con 0x7fc6d4100ff0 2024-04-25T13:12:55.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.982+0000 7fc6caffd640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc6c00a3c50 0x7fc6c00a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:55.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.983+0000 7fc6d9ce6640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc6c00a3c50 0x7fc6c00a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:55.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.983+0000 7fc6caffd640 1 -- 172.21.15.72:0/3567599820 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fc6c40f0be0 con 0x7fc6d4100ff0 2024-04-25T13:12:55.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.983+0000 7fc6d9ce6640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc6c00a3c50 0x7fc6c00a6110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0005fd0 tx=0x7fc6d0005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:55.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:55.987+0000 7fc6caffd640 1 -- 172.21.15.72:0/3567599820 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6c4003e60 con 0x7fc6d4100ff0 2024-04-25T13:12:56.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.215+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fc6d4062b50 con 0x7fc6d4100ff0 2024-04-25T13:12:56.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.297+0000 7fc6caffd640 1 -- 172.21.15.72:0/3567599820 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fc6c40bd5b0 con 0x7fc6d4100ff0 2024-04-25T13:12:56.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc6c00a3c50 msgr2=0x7fc6c00a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc6c00a3c50 0x7fc6c00a6110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0005fd0 tx=0x7fc6d0005ea0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 msgr2=0x7fc6d419ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 0x7fc6d419ed10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006010 tx=0x7fc6c4004430 comp rx=0 tx=0).stop 2024-04-25T13:12:56.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 shutdown_connections 2024-04-25T13:12:56.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fc6c00a3c50 0x7fc6c00a6110 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d41019b0 0x7fc6d41a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d4100ff0 0x7fc6d419ed10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 --2- 172.21.15.72:0/3567599820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6d4100290 0x7fc6d419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.299+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 >> 172.21.15.72:0/3567599820 conn(0x7fc6d40fbbc0 msgr2=0x7fc6d4102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:56.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.302+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 shutdown_connections 2024-04-25T13:12:56.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.302+0000 7fc6dbf4a640 1 -- 172.21.15.72:0/3567599820 wait complete. 2024-04-25T13:12:56.318 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T13:12:56.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.421+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1491875703 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 msgr2=0x7f3a64101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1491875703 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 0x7f3a64101280 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009ca0 tx=0x7f3a5c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1491875703 shutdown_connections 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1491875703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a64102a00 0x7f3a64109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1491875703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64102040 0x7f3a641024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1491875703 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 0x7f3a64101280 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1491875703 >> 172.21.15.72:0/1491875703 conn(0x7f3a640fc420 msgr2=0x7f3a640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1491875703 shutdown_connections 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1491875703 wait complete. 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 Processor -- start 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 -- start start 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 0x7f3a6406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:56.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a64102040 0x7f3a6419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64102a00 0x7f3a6419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410c9f0 con 0x7f3a64100e60 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410c870 con 0x7f3a64102a00 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.422+0000 7f3a6bf83640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410cb70 con 0x7f3a64102040 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a69d1f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 0x7f3a6406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a69d1f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 0x7f3a6406a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37262/0 (socket says 172.21.15.72:37262) 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a69d1f640 1 -- 172.21.15.72:0/1946000003 learned_addr learned my addr 172.21.15.72:0/1946000003 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a6a520640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64102a00 0x7f3a6419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a6951e640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a64102040 0x7f3a6419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:56.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a69d1f640 1 -- 172.21.15.72:0/1946000003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a64102040 msgr2=0x7f3a6419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a69d1f640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a64102040 0x7f3a6419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a69d1f640 1 -- 172.21.15.72:0/1946000003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64102a00 msgr2=0x7f3a6419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a69d1f640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64102a00 0x7f3a6419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a69d1f640 1 -- 172.21.15.72:0/1946000003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6419fdb0 con 0x7f3a64100e60 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a69d1f640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 0x7f3a6406a170 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009430 tx=0x7f3a5c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a5affd640 1 -- 172.21.15.72:0/1946000003 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c003b70 con 0x7f3a64100e60 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a5affd640 1 -- 172.21.15.72:0/1946000003 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a5c003d10 con 0x7f3a64100e60 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6419ffe0 con 0x7f3a64100e60 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a5affd640 1 -- 172.21.15.72:0/1946000003 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c02fb60 con 0x7f3a64100e60 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.423+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a641a0440 con 0x7f3a64100e60 2024-04-25T13:12:56.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.424+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a64108200 con 0x7f3a64100e60 2024-04-25T13:12:56.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.425+0000 7f3a5affd640 1 -- 172.21.15.72:0/1946000003 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f3a5c02fd00 con 0x7f3a64100e60 2024-04-25T13:12:56.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.426+0000 7f3a5affd640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3a500a3ca0 0x7f3a500a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:56.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.426+0000 7f3a6951e640 1 -- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3a500a3ca0 msgr2=0x7f3a500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/3348457169 2024-04-25T13:12:56.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.426+0000 7f3a5affd640 1 -- 172.21.15.72:0/1946000003 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f3a5c0f1860 con 0x7f3a64100e60 2024-04-25T13:12:56.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.426+0000 7f3a6951e640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3a500a3ca0 0x7f3a500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:12:56.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.431+0000 7f3a5affd640 1 -- 172.21.15.72:0/1946000003 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a5c0be230 con 0x7f3a64100e60 2024-04-25T13:12:56.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:56 smithi164 ceph-mon[32418]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:56.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:56 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3607587535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:56.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:56 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3567599820' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:12:56.557 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: ignoring --setuser ceph since I am not root 2024-04-25T13:12:56.557 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: ignoring --setgroup ceph since I am not root 2024-04-25T13:12:56.557 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:56 smithi164 ceph-mgr[33842]: -- 172.21.15.164:0/2556874029 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ab0c6c5a0 con 0x561ab0c40c00 2024-04-25T13:12:56.557 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:56.496+0000 7f42845bc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:12:56.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.616+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f3a6402d5c0 con 0x7f3a64100e60 2024-04-25T13:12:56.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.621+0000 7f3a5affd640 1 -- 172.21.15.72:0/1946000003 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f3a5c0bdb30 con 0x7f3a64100e60 2024-04-25T13:12:56.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3a500a3ca0 msgr2=0x7f3a500a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:12:56.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3a500a3ca0 0x7f3a500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 msgr2=0x7f3a6406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 0x7f3a6406a170 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009430 tx=0x7f3a5c0316e0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 shutdown_connections 2024-04-25T13:12:56.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f3a500a3ca0 0x7f3a500a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64102a00 0x7f3a6419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a64102040 0x7f3a6419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 --2- 172.21.15.72:0/1946000003 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a64100e60 0x7f3a6406a170 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 >> 172.21.15.72:0/1946000003 conn(0x7f3a640fc420 msgr2=0x7f3a641053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:56.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 shutdown_connections 2024-04-25T13:12:56.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.623+0000 7f3a6bf83640 1 -- 172.21.15.72:0/1946000003 wait complete. 2024-04-25T13:12:56.633 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:56 smithi072 ceph-mon[29199]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:56.633 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:56 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3607587535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:56.634 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:56 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3567599820' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:12:56.634 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:56 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: ignoring --setuser ceph since I am not root 2024-04-25T13:12:56.634 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:56 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: ignoring --setgroup ceph since I am not root 2024-04-25T13:12:56.634 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:56 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:56.482+0000 7f103737d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:12:56.635 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T13:12:56.658 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:56 smithi193 ceph-mon[32743]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:56.659 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:56 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3607587535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:12:56.659 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:56 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3567599820' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:12:56.659 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: ignoring --setuser ceph since I am not root 2024-04-25T13:12:56.659 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: ignoring --setgroup ceph since I am not root 2024-04-25T13:12:56.659 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:56 smithi193 ceph-mgr[34644]: -- 172.21.15.193:0/3725962685 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad170565a0 con 0x55ad1702b000 2024-04-25T13:12:56.659 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:56.477+0000 7f5ae97e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:12:56.659 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:56.658+0000 7f5ae97e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:12:56.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 -- 172.21.15.72:0/2517324307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 msgr2=0x7fea841024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 --2- 172.21.15.72:0/2517324307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 0x7fea841024c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fea78009900 tx=0x7fea7802a410 comp rx=0 tx=0).stop 2024-04-25T13:12:56.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 -- 172.21.15.72:0/2517324307 shutdown_connections 2024-04-25T13:12:56.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 --2- 172.21.15.72:0/2517324307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea84102a00 0x7fea84109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 --2- 172.21.15.72:0/2517324307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 0x7fea841024c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 --2- 172.21.15.72:0/2517324307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea84100e60 0x7fea84101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 -- 172.21.15.72:0/2517324307 >> 172.21.15.72:0/2517324307 conn(0x7fea840fc420 msgr2=0x7fea840fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 -- 172.21.15.72:0/2517324307 shutdown_connections 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 -- 172.21.15.72:0/2517324307 wait complete. 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.736+0000 7fea890cb640 1 Processor -- start 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea890cb640 1 -- start start 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea890cb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea84100e60 0x7fea84197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea890cb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 0x7fea84197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea890cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea84102a00 0x7fea8419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea890cb640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fea8410c9f0 con 0x7fea84102040 2024-04-25T13:12:56.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea890cb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fea8410c870 con 0x7fea84100e60 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea890cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea8410cb70 con 0x7fea84102a00 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea8259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 0x7fea84197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea8259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 0x7fea84197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37278/0 (socket says 172.21.15.72:37278) 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea8259c640 1 -- 172.21.15.72:0/3802949768 learned_addr learned my addr 172.21.15.72:0/3802949768 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea82d9d640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea84100e60 0x7fea84197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea8359e640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea84102a00 0x7fea8419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea82d9d640 1 -- 172.21.15.72:0/3802949768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea84102a00 msgr2=0x7fea8419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea82d9d640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea84102a00 0x7fea8419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea82d9d640 1 -- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 msgr2=0x7fea84197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea82d9d640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 0x7fea84197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea82d9d640 1 -- 172.21.15.72:0/3802949768 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8419ec60 con 0x7fea84100e60 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea8259c640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 0x7fea84197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.737+0000 7fea82d9d640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea84100e60 0x7fea84197040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea7400e9c0 tx=0x7fea7400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:56.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.738+0000 7fea6bfff640 1 -- 172.21.15.72:0/3802949768 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7400ccc0 con 0x7fea84100e60 2024-04-25T13:12:56.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.738+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8419ef50 con 0x7fea84100e60 2024-04-25T13:12:56.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.738+0000 7fea6bfff640 1 -- 172.21.15.72:0/3802949768 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea740035f0 con 0x7fea84100e60 2024-04-25T13:12:56.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.738+0000 7fea6bfff640 1 -- 172.21.15.72:0/3802949768 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7400ce60 con 0x7fea84100e60 2024-04-25T13:12:56.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.738+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8419f3b0 con 0x7fea84100e60 2024-04-25T13:12:56.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.739+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea84108200 con 0x7fea84100e60 2024-04-25T13:12:56.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.740+0000 7fea6bfff640 1 -- 172.21.15.72:0/3802949768 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fea74004000 con 0x7fea84100e60 2024-04-25T13:12:56.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.740+0000 7fea6bfff640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fea6c0a3ca0 0x7fea6c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:56.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.740+0000 7fea8259c640 1 -- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fea6c0a3ca0 msgr2=0x7fea6c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/3348457169 2024-04-25T13:12:56.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.741+0000 7fea6bfff640 1 -- 172.21.15.72:0/3802949768 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fea740d0a20 con 0x7fea84100e60 2024-04-25T13:12:56.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.741+0000 7fea8259c640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fea6c0a3ca0 0x7fea6c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:12:56.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.744+0000 7fea6bfff640 1 -- 172.21.15.72:0/3802949768 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea74043e20 con 0x7fea84100e60 2024-04-25T13:12:56.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.926+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fea8402d5c0 con 0x7fea84100e60 2024-04-25T13:12:56.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.933+0000 7fea6bfff640 1 -- 172.21.15.72:0/3802949768 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7fea7409d3f0 con 0x7fea84100e60 2024-04-25T13:12:56.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fea6c0a3ca0 msgr2=0x7fea6c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:12:56.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fea6c0a3ca0 0x7fea6c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea84100e60 msgr2=0x7fea84197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea84100e60 0x7fea84197040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea7400e9c0 tx=0x7fea7400ee80 comp rx=0 tx=0).stop 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 shutdown_connections 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fea6c0a3ca0 0x7fea6c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea84102a00 0x7fea8419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea84102040 0x7fea84197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 --2- 172.21.15.72:0/3802949768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea84100e60 0x7fea84197040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 >> 172.21.15.72:0/3802949768 conn(0x7fea840fc420 msgr2=0x7fea841053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 shutdown_connections 2024-04-25T13:12:56.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:56.935+0000 7fea890cb640 1 -- 172.21.15.72:0/3802949768 wait complete. 2024-04-25T13:12:56.945 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:56 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:56.672+0000 7f103737d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:12:56.945 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:56 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:56.780+0000 7f103737d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:12:56.946 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T13:12:56.975 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T13:12:56.975 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T13:12:56.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:12:56.975 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-04-25T13:12:56.981 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:56.759+0000 7f5ae97e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:12:56.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:12:56.991 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2024-04-25T13:12:57.056 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:56.681+0000 7f42845bc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:12:57.056 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:56.785+0000 7f42845bc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:12:57.057 INFO:teuthology.orchestra.run.smithi072.stderr:Cannot find device "ceph-brx" 2024-04-25T13:12:57.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:12:57.058 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T13:12:57.058 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:12:57.058 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:12:57.059 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T13:12:57.274 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:57.096+0000 7f103737d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:12:57.274 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:57.186+0000 7f103737d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:12:57.352 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:57.481 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:57.062+0000 7f5ae97e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:12:57.481 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:57.145+0000 7f5ae97e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:12:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:57 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3567599820' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:12:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:57 smithi193 ceph-mon[32743]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T13:12:57.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:57 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1946000003' entity='client.admin' 2024-04-25T13:12:57.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:57 smithi193 ceph-mon[32743]: from='client.? ' entity='client.admin' 2024-04-25T13:12:57.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:57 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3567599820' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:12:57.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:57 smithi164 ceph-mon[32418]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T13:12:57.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:57 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1946000003' entity='client.admin' 2024-04-25T13:12:57.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:57 smithi164 ceph-mon[32418]: from='client.? ' entity='client.admin' 2024-04-25T13:12:57.557 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:57.094+0000 7f42845bc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:12:57.557 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:57.179+0000 7f42845bc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:12:57.597 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:57 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3567599820' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:12:57.597 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:57 smithi072 ceph-mon[29199]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T13:12:57.597 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:57 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1946000003' entity='client.admin' 2024-04-25T13:12:57.597 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:57 smithi072 ceph-mon[29199]: from='client.? ' entity='client.admin' 2024-04-25T13:12:58.523 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:58 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:58.418+0000 7f103737d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:12:58.648 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:58.313+0000 7f5ae97e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:12:58.648 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:58.479+0000 7f5ae97e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:12:58.648 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:58.647+0000 7f5ae97e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:12:58.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:58 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:58.368+0000 7f42845bc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:12:58.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:58 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:58.538+0000 7f42845bc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:12:58.926 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:58 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:58.594+0000 7f103737d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:12:58.927 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:58 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:58.771+0000 7f103737d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:12:58.997 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:58 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:58.709+0000 7f42845bc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:12:59.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 -- 172.21.15.72:0/4170997302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa108107990 msgr2=0x7fa108107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:59.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 --2- 172.21.15.72:0/4170997302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa108107990 0x7fa108107d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa104009990 tx=0x7fa104009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 -- 172.21.15.72:0/4170997302 shutdown_connections 2024-04-25T13:12:59.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 --2- 172.21.15.72:0/4170997302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108102300 0x7fa10810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 --2- 172.21.15.72:0/4170997302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa108107990 0x7fa108107d70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 --2- 172.21.15.72:0/4170997302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa108101940 0x7fa108101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 -- 172.21.15.72:0/4170997302 >> 172.21.15.72:0/4170997302 conn(0x7fa1080788f0 msgr2=0x7fa108078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:59.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 -- 172.21.15.72:0/4170997302 shutdown_connections 2024-04-25T13:12:59.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 -- 172.21.15.72:0/4170997302 wait complete. 2024-04-25T13:12:59.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.161+0000 7fa11019f700 1 Processor -- start 2024-04-25T13:12:59.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa11019f700 1 -- start start 2024-04-25T13:12:59.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa11019f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa108101940 0x7fa108196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:59.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa11019f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa108102300 0x7fa1081973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:59.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa11019f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108107990 0x7fa10819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:59.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa11019f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa108110a00 con 0x7fa108102300 2024-04-25T13:12:59.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa11019f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa108110880 con 0x7fa108107990 2024-04-25T13:12:59.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa11019f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa108110b80 con 0x7fa108101940 2024-04-25T13:12:59.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10d73a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa108102300 0x7fa1081973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:59.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10d73a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa108102300 0x7fa1081973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37294/0 (socket says 172.21.15.72:37294) 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10d73a700 1 -- 172.21.15.72:0/4167547996 learned_addr learned my addr 172.21.15.72:0/4167547996 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10df3b700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa108101940 0x7fa108196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10e73c700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108107990 0x7fa10819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10d73a700 1 -- 172.21.15.72:0/4167547996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa108101940 msgr2=0x7fa108196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10d73a700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa108101940 0x7fa108196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10d73a700 1 -- 172.21.15.72:0/4167547996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108107990 msgr2=0x7fa10819b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10d73a700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108107990 0x7fa10819b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10d73a700 1 -- 172.21.15.72:0/4167547996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10819bec0 con 0x7fa108102300 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10df3b700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa108101940 0x7fa108196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.162+0000 7fa10d73a700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa108102300 0x7fa1081973f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800d950 tx=0x7fa0f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.163+0000 7fa0feffd700 1 -- 172.21.15.72:0/4167547996 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8016070 con 0x7fa108102300 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.163+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10819c1b0 con 0x7fa108102300 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.163+0000 7fa0feffd700 1 -- 172.21.15.72:0/4167547996 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0f800fd40 con 0x7fa108102300 2024-04-25T13:12:59.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.163+0000 7fa0feffd700 1 -- 172.21.15.72:0/4167547996 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f800e3b0 con 0x7fa108102300 2024-04-25T13:12:59.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.163+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a3a30 con 0x7fa108102300 2024-04-25T13:12:59.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.164+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa108109eb0 con 0x7fa108102300 2024-04-25T13:12:59.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.165+0000 7fa0feffd700 1 -- 172.21.15.72:0/4167547996 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fa0f8019070 con 0x7fa108102300 2024-04-25T13:12:59.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.165+0000 7fa0feffd700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0f40a3c90 0x7fa0f40a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:59.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.165+0000 7fa10df3b700 1 -- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0f40a3c90 msgr2=0x7fa0f40a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/3348457169 2024-04-25T13:12:59.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.165+0000 7fa0feffd700 1 -- 172.21.15.72:0/4167547996 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa0f80cfea0 con 0x7fa108102300 2024-04-25T13:12:59.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.166+0000 7fa10df3b700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0f40a3c90 0x7fa0f40a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:12:59.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.169+0000 7fa0feffd700 1 -- 172.21.15.72:0/4167547996 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0f809c870 con 0x7fa108102300 2024-04-25T13:12:59.212 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:59 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:59.073+0000 7f103737d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:12:59.212 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:59 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:59.173+0000 7f103737d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:12:59.222 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:58.931+0000 7f5ae97e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:12:59.223 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:59.022+0000 7f5ae97e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:12:59.294 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:58 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:58.997+0000 7f42845bc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:12:59.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:59.090+0000 7f42845bc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:12:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.367+0000 7fa10df3b700 1 -- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0f40a3c90 msgr2=0x7fa0f40a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/3348457169 2024-04-25T13:12:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.367+0000 7fa10df3b700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0f40a3c90 0x7fa0f40a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:12:59.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.424+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa108069730 con 0x7fa108102300 2024-04-25T13:12:59.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.424+0000 7fa0feffd700 1 -- 172.21.15.72:0/4167547996 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7fa0f809c170 con 0x7fa108102300 2024-04-25T13:12:59.426 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:12:59.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0f40a3c90 msgr2=0x7fa0f40a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:12:59.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0f40a3c90 0x7fa0f40a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa108102300 msgr2=0x7fa1081973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:59.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa108102300 0x7fa1081973f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800d950 tx=0x7fa0f800dc60 comp rx=0 tx=0).stop 2024-04-25T13:12:59.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 shutdown_connections 2024-04-25T13:12:59.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108107990 0x7fa10819b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa108101940 0x7fa108196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0f40a3c90 0x7fa0f40a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 --2- 172.21.15.72:0/4167547996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa108102300 0x7fa1081973f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:59.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 >> 172.21.15.72:0/4167547996 conn(0x7fa1080788f0 msgr2=0x7fa108105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:59.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 shutdown_connections 2024-04-25T13:12:59.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:12:59.427+0000 7fa11019f700 1 -- 172.21.15.72:0/4167547996 wait complete. 2024-04-25T13:12:59.476 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:59 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:59.384+0000 7f103737d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:12:59.476 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:59.223+0000 7f5ae97e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:12:59.477 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:12:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:12:59.415+0000 7f5ae97e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:12:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:12:59 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/4167547996' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:12:59.557 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:59.294+0000 7f42845bc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:12:59.557 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:12:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:12:59.490+0000 7f42845bc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:12:59.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:12:59 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/4167547996' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:12:59.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:12:59 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/4167547996' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:12:59.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:12:59 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:12:59.589+0000 7f103737d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:13:00.208 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:13:00.208 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:13:00.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2024-04-25T13:13:00.208 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:13:00.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:13:00.208 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:13:00.208 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:13:00.208 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:13:00.208 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:13:00.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:00.209 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2024-04-25T13:13:00.224 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:13:00.224 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f0 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f1 2024-04-25T13:13:00.225 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:13:00.226 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:13:00.226 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:13:00.226 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:13:00.226 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T13:13:00.226 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 42012sec preferred_lft 42012sec 2024-04-25T13:13:00.226 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::4cad:bb47:f4aa:acd5/64 scope link noprefixroute 2024-04-25T13:13:00.226 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:13:00.226 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T13:13:00.227 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:13:00.227 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-25T13:13:00.227 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add name ceph-brx type bridge 2024-04-25T13:13:00.227 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr flush dev ceph-brx 2024-04-25T13:13:00.227 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set ceph-brx up 2024-04-25T13:13:00.227 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T13:13:00.227 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-25T13:13:00.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:13:00.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:13:00.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:00.421 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:13:00.422 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:00.008+0000 7f5ae97e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:13:00.433 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:00.225+0000 7f103737d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:13:00.449 INFO:teuthology.orchestra.run.smithi072.stdout:1 2024-04-25T13:13:00.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:00.450 DEBUG:teuthology.orchestra.run.smithi072:> route 2024-04-25T13:13:00.520 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:00 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:00.094+0000 7f42845bc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:13:00.522 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2024-04-25T13:13:00.522 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:13:00.523 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:13:00.523 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:13:00.523 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:13:00.523 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:13:00.523 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-25T13:13:00.523 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:13:00.523 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:13:00.524 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:13:00.524 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-25T13:13:00.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:13:00.713 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:00.422+0000 7f5ae97e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:13:00.714 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:00.517+0000 7f5ae97e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:13:00.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:13:00.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:00.785 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-04-25T13:13:00.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:00.799 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list-id 2024-04-25T13:13:00.806 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:00 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:00.520+0000 7f42845bc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:13:00.806 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:00 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:00.617+0000 7f42845bc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:13:00.854 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:13:00.854 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-25T13:13:00.854 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:13:00.854 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T13:13:00.855 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-25T13:13:00.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:13:00.949 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:00.690+0000 7f103737d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:13:00.949 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:00.797+0000 7f103737d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:13:00.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:13:00.971 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T13:13:00.971 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:13:00.971 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-25T13:13:00.971 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T13:13:00.971 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T13:13:00.971 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T13:13:00.971 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T13:13:00.971 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T13:13:00.971 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-25T13:13:00.980 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:00.713+0000 7f5ae97e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:13:01.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:13:01.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:13:01.095 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:13:01.096 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-25T13:13:01.096 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.0 up 2024-04-25T13:13:01.096 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T13:13:01.096 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-25T13:13:01.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:13:01.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:13:01.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:13:01.174 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:13:01.189 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:13:01.189 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:13:01.189 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:13:01.246 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T13:13:01.247 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T13:13:01.247 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/b84445af-f374-4c76-ad5e-e65b51e37dce 2024-04-25T13:13:01.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:01.247 DEBUG:teuthology.orchestra.run.smithi072:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/b84445af-f374-4c76-ad5e-e65b51e37dce /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T13:13:01.275 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:01.014+0000 7f103737d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:13:01.307 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:00 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:00.820+0000 7f42845bc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:13:01.324 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T13:13:01.324 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T13:13:01.325 INFO:teuthology.orchestra.run.smithi072.stdout:invalid new device string format 2024-04-25T13:13:01.434 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T13:13:01.434 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T13:13:01.434 INFO:teuthology.orchestra.run.smithi072.stdout:invalid new device string format 2024-04-25T13:13:01.434 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.72:3300,172.21.15.164:3300,172.21.15.193:3300" 2024-04-25T13:13:01.434 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.72:3300,172.21.15.164:3300,172.21.15.193:3300:/volumes/qa/sv_1/b84445af-f374-4c76-ad5e-e65b51e37dce 2024-04-25T13:13:01.435 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=d8ca43ea-0303-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T13:13:01.435 INFO:teuthology.orchestra.run.smithi072.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T13:13:01.436 INFO:teuthology.orchestra.run.smithi072.stdout: You just mounted a file system that supports labels which does not 2024-04-25T13:13:01.436 INFO:teuthology.orchestra.run.smithi072.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T13:13:01.436 INFO:teuthology.orchestra.run.smithi072.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T13:13:01.436 INFO:teuthology.orchestra.run.smithi072.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T13:13:01.438 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T13:13:01.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:01.438 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:13:01.464 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:13:01.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:13:01.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:13:01.561 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:01.231+0000 7f5ae97e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:13:01.561 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:01.322+0000 7f5ae97e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:13:01.562 INFO:teuthology.orchestra.run.smithi072.stdout:client24869 2024-04-25T13:13:01.563 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:13:01.563 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:13:01.579 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:13:01.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:13:01.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:13:01.677 INFO:teuthology.orchestra.run.smithi072.stdout:client24869 2024-04-25T13:13:01.677 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:13:01.685 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:01 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:01.346+0000 7f42845bc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:13:01.685 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:01 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:01.440+0000 7f42845bc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:13:01.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:13:01.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:13:01.734 INFO:teuthology.orchestra.run.smithi072.stdout:client24869 2024-04-25T13:13:01.734 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968.client24869/status') 2024-04-25T13:13:01.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:13:01.791 INFO:teuthology.orchestra.run.smithi072.stdout:instance: client.24869 (3)172.21.15.72:0/669967807 2024-04-25T13:13:01.792 INFO:teuthology.orchestra.run.smithi072.stdout:blocklisted: false 2024-04-25T13:13:01.792 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records in 2024-04-25T13:13:01.792 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records out 2024-04-25T13:13:01.792 INFO:teuthology.orchestra.run.smithi072.stderr:70 bytes copied, 2.671e-05 s, 2.6 MB/s 2024-04-25T13:13:01.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:13:01.799 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:13:01.808 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:13:01.808 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:13:01.808 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T13:13:01.828 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:01.578+0000 7f103737d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:13:01.828 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:01.678+0000 7f103737d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:13:01.898 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:01.561+0000 7f5ae97e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:13:01.898 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:01.692+0000 7f5ae97e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:13:01.898 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:01.785+0000 7f5ae97e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:13:01.898 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:01.898+0000 7f5ae97e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:13:01.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 -- 172.21.15.72:0/4218650656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018100e60 msgr2=0x7fa018101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:01.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/4218650656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018100e60 0x7fa018101280 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa008009900 tx=0x7fa00802a410 comp rx=0 tx=0).stop 2024-04-25T13:13:01.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 -- 172.21.15.72:0/4218650656 shutdown_connections 2024-04-25T13:13:01.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/4218650656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa018102a00 0x7fa018109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:01.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/4218650656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa018102040 0x7fa0181024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:01.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/4218650656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018100e60 0x7fa018101280 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:01.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 -- 172.21.15.72:0/4218650656 >> 172.21.15.72:0/4218650656 conn(0x7fa0180fc420 msgr2=0x7fa0180fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 -- 172.21.15.72:0/4218650656 shutdown_connections 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 -- 172.21.15.72:0/4218650656 wait complete. 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 Processor -- start 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.910+0000 7fa01dd5d640 1 -- start start 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa01dd5d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa018100e60 0x7fa01806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa01dd5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa018102040 0x7fa01819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa01dd5d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018102a00 0x7fa01819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa01dd5d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa01810ca40 con 0x7fa018102a00 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa01dd5d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa01810c8c0 con 0x7fa018100e60 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa01dd5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa01810cbc0 con 0x7fa018102040 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa017fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018102a00 0x7fa01819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa017fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018102a00 0x7fa01819f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37302/0 (socket says 172.21.15.72:37302) 2024-04-25T13:13:01.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa017fff640 1 -- 172.21.15.72:0/1811423378 learned_addr learned my addr 172.21.15.72:0/1811423378 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa00ffff640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa018102040 0x7fa01819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa0177fe640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa018100e60 0x7fa01806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa017fff640 1 -- 172.21.15.72:0/1811423378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa018102040 msgr2=0x7fa01819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa017fff640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa018102040 0x7fa01819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa017fff640 1 -- 172.21.15.72:0/1811423378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa018100e60 msgr2=0x7fa01806a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa017fff640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa018100e60 0x7fa01806a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa017fff640 1 -- 172.21.15.72:0/1811423378 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01819fde0 con 0x7fa018102a00 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa00ffff640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa018102040 0x7fa01819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa017fff640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018102a00 0x7fa01819f4d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa010009fd0 tx=0x7fa01000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.911+0000 7fa0157fa640 1 -- 172.21.15.72:0/1811423378 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010017070 con 0x7fa018102a00 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.912+0000 7fa0157fa640 1 -- 172.21.15.72:0/1811423378 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa010003e20 con 0x7fa018102a00 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.912+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0181a0070 con 0x7fa018102a00 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.912+0000 7fa0157fa640 1 -- 172.21.15.72:0/1811423378 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010010460 con 0x7fa018102a00 2024-04-25T13:13:01.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.912+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0181a0570 con 0x7fa018102a00 2024-04-25T13:13:01.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.913+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa018108200 con 0x7fa018102a00 2024-04-25T13:13:01.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.914+0000 7fa0157fa640 1 -- 172.21.15.72:0/1811423378 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fa010003970 con 0x7fa018102a00 2024-04-25T13:13:01.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.914+0000 7fa0157fa640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0040a3ca0 0x7fa0040a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:01.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.915+0000 7fa0177fe640 1 -- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0040a3ca0 msgr2=0x7fa0040a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/3348457169 2024-04-25T13:13:01.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.915+0000 7fa0157fa640 1 -- 172.21.15.72:0/1811423378 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa0100d0170 con 0x7fa018102a00 2024-04-25T13:13:01.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.915+0000 7fa0177fe640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0040a3ca0 0x7fa0040a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:13:01.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:01.919+0000 7fa0157fa640 1 -- 172.21.15.72:0/1811423378 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa01009c4f0 con 0x7fa018102a00 2024-04-25T13:13:02.032 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:01 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:01.685+0000 7f42845bc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:13:02.032 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:01 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:01.820+0000 7f42845bc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:13:02.032 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:01 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:01.917+0000 7f42845bc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:13:02.033 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:02.032+0000 7f42845bc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:13:02.086 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:01.945+0000 7f103737d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:13:02.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.116+0000 7fa0177fe640 1 -- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0040a3ca0 msgr2=0x7fa0040a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/3348457169 2024-04-25T13:13:02.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.116+0000 7fa0177fe640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0040a3ca0 0x7fa0040a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:13:02.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.142+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fa018062b50 con 0x7fa018102a00 2024-04-25T13:13:02.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.194+0000 7fa0157fa640 1 -- 172.21.15.72:0/1811423378 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v21) ==== 118+0+0 (secure 0 0 0) 0x7fa0100a07f0 con 0x7fa018102a00 2024-04-25T13:13:02.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.196+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0040a3ca0 msgr2=0x7fa0040a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:13:02.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.196+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0040a3ca0 0x7fa0040a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.196+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018102a00 msgr2=0x7fa01819f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:02.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.196+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018102a00 0x7fa01819f4d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa010009fd0 tx=0x7fa01000b260 comp rx=0 tx=0).stop 2024-04-25T13:13:02.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.196+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 shutdown_connections 2024-04-25T13:13:02.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.196+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7fa0040a3ca0 0x7fa0040a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.196+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa018102a00 0x7fa01819f4d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.196+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa018102040 0x7fa01819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.196+0000 7fa01dd5d640 1 --2- 172.21.15.72:0/1811423378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa018100e60 0x7fa01806a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.197+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 >> 172.21.15.72:0/1811423378 conn(0x7fa0180fc420 msgr2=0x7fa0181053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:02.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.197+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 shutdown_connections 2024-04-25T13:13:02.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.197+0000 7fa01dd5d640 1 -- 172.21.15.72:0/1811423378 wait complete. 2024-04-25T13:13:02.208 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T13:13:02.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:02 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1811423378' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T13:13:02.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 -- 172.21.15.72:0/553748544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368102040 msgr2=0x7f13681024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:02.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 --2- 172.21.15.72:0/553748544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368102040 0x7f13681024c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f135c009900 tx=0x7f135c02a410 comp rx=0 tx=0).stop 2024-04-25T13:13:02.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 -- 172.21.15.72:0/553748544 shutdown_connections 2024-04-25T13:13:02.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 --2- 172.21.15.72:0/553748544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1368102a00 0x7f1368109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 --2- 172.21.15.72:0/553748544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368102040 0x7f13681024c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 --2- 172.21.15.72:0/553748544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1368100e60 0x7f1368101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 -- 172.21.15.72:0/553748544 >> 172.21.15.72:0/553748544 conn(0x7f13680fc420 msgr2=0x7f13680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 -- 172.21.15.72:0/553748544 shutdown_connections 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 -- 172.21.15.72:0/553748544 wait complete. 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 Processor -- start 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 -- start start 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368100e60 0x7f136806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1368102040 0x7f136819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1368102a00 0x7f136819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f136810c9f0 con 0x7f1368100e60 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.353+0000 7f136f36f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f136810c870 con 0x7f1368102040 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136f36f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f136810cb70 con 0x7f1368102a00 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d10b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368100e60 0x7f136806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:02.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d10b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368100e60 0x7f136806a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37312/0 (socket says 172.21.15.72:37312) 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d10b640 1 -- 172.21.15.72:0/3027070020 learned_addr learned my addr 172.21.15.72:0/3027070020 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136c90a640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1368102040 0x7f136819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d90c640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1368102a00 0x7f136819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d10b640 1 -- 172.21.15.72:0/3027070020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1368102a00 msgr2=0x7f136819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d10b640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1368102a00 0x7f136819f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d10b640 1 -- 172.21.15.72:0/3027070020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1368102040 msgr2=0x7f136819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d10b640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1368102040 0x7f136819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d10b640 1 -- 172.21.15.72:0/3027070020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136819fe40 con 0x7f1368100e60 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136d10b640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368100e60 0x7f136806a170 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f136000ea80 tx=0x7f136000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136000ce40 con 0x7f1368100e60 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13681a00d0 con 0x7f1368100e60 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1360011920 con 0x7f1368100e60 2024-04-25T13:13:02.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136001aaa0 con 0x7f1368100e60 2024-04-25T13:13:02.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.354+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13681a05f0 con 0x7f1368100e60 2024-04-25T13:13:02.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.355+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1368108270 con 0x7f1368100e60 2024-04-25T13:13:02.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.357+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f136001ac40 con 0x7f1368100e60 2024-04-25T13:13:02.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.358+0000 7f13667fc640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f13500a3ca0 0x7f13500a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:02.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.358+0000 7f136c90a640 1 -- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f13500a3ca0 msgr2=0x7f13500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/3348457169 2024-04-25T13:13:02.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.358+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f13600d0980 con 0x7f1368100e60 2024-04-25T13:13:02.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.358+0000 7f136c90a640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f13500a3ca0 0x7f13500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:13:02.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.363+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1360016030 con 0x7f1368100e60 2024-04-25T13:13:02.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:02 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1811423378' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T13:13:02.481 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:02.200+0000 7f5ae97e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:13:02.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:02 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1811423378' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T13:13:02.524 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:02.085+0000 7f103737d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:13:02.524 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:02.185+0000 7f103737d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:13:02.524 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:02.311+0000 7f103737d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:13:02.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.559+0000 7f136c90a640 1 -- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f13500a3ca0 msgr2=0x7f13500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/3348457169 2024-04-25T13:13:02.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.559+0000 7f136c90a640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f13500a3ca0 0x7f13500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:13:02.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.574+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 --> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f13680624f0 con 0x7f13500a3ca0 2024-04-25T13:13:02.806 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:02.337+0000 7f42845bc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:13:02.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.960+0000 7f136c90a640 1 -- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f13500a3ca0 msgr2=0x7f13500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/3348457169 2024-04-25T13:13:02.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:02.960+0000 7f136c90a640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f13500a3ca0 0x7f13500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:13:03.023 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:02.625+0000 7f103737d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:13:03.201 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:02.795+0000 7f5ae97e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:13:03.201 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z[34640]: 2024-04-25T13:13:03.006+0000 7f5ae97e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:13:03.202 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:02.939+0000 7f42845bc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:13:03.202 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y[33838]: 2024-04-25T13:13:03.154+0000 7f42845bc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:13:03.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:03.216+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f13600992c0 con 0x7f1368100e60 2024-04-25T13:13:03.462 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.241+0000 7f103737d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:13:03.462 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1811423378' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T13:13:03.462 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:13:03.462 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: Standby manager daemon z restarted 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: Standby manager daemon z started 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: Standby manager daemon y restarted 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: Standby manager daemon y started 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:13:03.463 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-mon[29199]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:13:03.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:03.474+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f136009a920 con 0x7f1368100e60 2024-04-25T13:13:03.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:03.475+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f13500a3ca0 msgr2=0x7f13500a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:13:03.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:03.475+0000 7f13667fc640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3348457169,v1:172.21.15.72:6801/3348457169] conn(0x7f13500a3ca0 0x7f13500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1811423378' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: Standby manager daemon z restarted 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: Standby manager daemon z started 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: Standby manager daemon y restarted 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: Standby manager daemon y started 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:13:03.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:13:03.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:13:03.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:03 smithi193 ceph-mon[32743]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1811423378' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: Standby manager daemon z restarted 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: Standby manager daemon z started 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.193:0/1759390688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: Standby manager daemon y restarted 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: Standby manager daemon y started 2024-04-25T13:13:03.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:13:03.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:13:03.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:13:03.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:03 smithi164 ceph-mon[32418]: from='mgr.? 172.21.15.164:0/2015637837' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:13:03.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.462+0000 7f103737d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:13:03.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f96cb1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:03.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f96cb1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:03.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f96cb1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:03.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f96cb1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:03.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f96cb1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:03.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f98cb5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:03.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f98cb5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:03.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f98cb5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:03.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f98cb5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:03.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:03.704+0000 7f0f98cb5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:04.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.481+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mon.0 v2:172.21.15.72:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f1360078cf0 con 0x7f1368100e60 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: Active manager daemon x restarted 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: Activating manager daemon x 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: mgrmap e21: x(active, starting, since 0.0124949s), standbys: y, z 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:13:04.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:13:04.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: Manager daemon x is now available 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:04 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:13:04.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.482+0000 7f13667fc640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1350035890 0x7f1350035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:04.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.482+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 --> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f13600a0df0 con 0x7f1350035890 2024-04-25T13:13:04.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.484+0000 7f136c90a640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1350035890 0x7f1350035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:04.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.486+0000 7f136c90a640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1350035890 0x7f1350035c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f135c000c00 tx=0x7f135c0314a0 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: Active manager daemon x restarted 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: Activating manager daemon x 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: mgrmap e21: x(active, starting, since 0.0124949s), standbys: y, z 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:13:04.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:13:04.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:13:04.496 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:13:04.497 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:13:04.497 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:13:04.497 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:13:04.497 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:13:04.497 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: Manager daemon x is now available 2024-04-25T13:13:04.497 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:04.497 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:13:04.497 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:13:04.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.503+0000 7f13667fc640 1 -- 172.21.15.72:0/3027070020 <== mgr.15276 v2:172.21.15.72:6800/3170992219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f13600a0df0 con 0x7f1350035890 2024-04-25T13:13:04.503 INFO:teuthology.orchestra.run.smithi072.stdout:{} 2024-04-25T13:13:04.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.505+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1350035890 msgr2=0x7f1350035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:04.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.505+0000 7f136f36f640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1350035890 0x7f1350035c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f135c000c00 tx=0x7f135c0314a0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.505+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368100e60 msgr2=0x7f136806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:04.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.505+0000 7f136f36f640 1 --2- 172.21.15.72:0/3027070020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368100e60 0x7f136806a170 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f136000ea80 tx=0x7f136000ef40 comp rx=0 tx=0).stop 2024-04-25T13:13:04.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.505+0000 7f136d90c640 1 -- 172.21.15.72:0/3027070020 reap_dead start 2024-04-25T13:13:04.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.505+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 shutdown_connections 2024-04-25T13:13:04.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.505+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 >> 172.21.15.72:0/3027070020 conn(0x7f13680fc420 msgr2=0x7f1368105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:04.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.505+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 shutdown_connections 2024-04-25T13:13:04.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.505+0000 7f136f36f640 1 -- 172.21.15.72:0/3027070020 wait complete. 2024-04-25T13:13:04.516 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T13:13:04.527 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:13:04.536 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:13:04.536 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:13:04.536 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T13:13:04.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-04-25T13:13:04.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: Active manager daemon x restarted 2024-04-25T13:13:04.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: Activating manager daemon x 2024-04-25T13:13:04.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:13:04.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: mgrmap e21: x(active, starting, since 0.0124949s), standbys: y, z 2024-04-25T13:13:04.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:13:04.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:13:04.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:13:04.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:13:04.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:13:04.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:13:04.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: Manager daemon x is now available 2024-04-25T13:13:04.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:04.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:13:04.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:04 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:13:04.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 -- 172.21.15.72:0/2869692381 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34105360 msgr2=0x7f1b34107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:04.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 --2- 172.21.15.72:0/2869692381 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34105360 0x7f1b34107750 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009860 tx=0x7f1b2802a340 comp rx=0 tx=0).stop 2024-04-25T13:13:04.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 -- 172.21.15.72:0/2869692381 shutdown_connections 2024-04-25T13:13:04.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 --2- 172.21.15.72:0/2869692381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b34107c90 0x7f1b3410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 --2- 172.21.15.72:0/2869692381 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34105360 0x7f1b34107750 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 --2- 172.21.15.72:0/2869692381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102a30 0x7f1b34104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 -- 172.21.15.72:0/2869692381 >> 172.21.15.72:0/2869692381 conn(0x7f1b340fc420 msgr2=0x7f1b340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:04.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 -- 172.21.15.72:0/2869692381 shutdown_connections 2024-04-25T13:13:04.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 -- 172.21.15.72:0/2869692381 wait complete. 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 Processor -- start 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 -- start start 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b34102a30 0x7f1b34198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.638+0000 7f1b39405640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34105360 0x7f1b34198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b39405640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34107c90 0x7f1b3419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b39405640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3410dab0 con 0x7f1b34107c90 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b39405640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3410d930 con 0x7f1b34105360 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b39405640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3410dc30 con 0x7f1b34102a30 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34107c90 0x7f1b3419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34107c90 0x7f1b3419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37456/0 (socket says 172.21.15.72:37456) 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b337fe640 1 -- 172.21.15.72:0/3113461433 learned_addr learned my addr 172.21.15.72:0/3113461433 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b327fc640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34105360 0x7f1b34198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:04.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b32ffd640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b34102a30 0x7f1b34198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b337fe640 1 -- 172.21.15.72:0/3113461433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b34102a30 msgr2=0x7f1b34198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b337fe640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b34102a30 0x7f1b34198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b337fe640 1 -- 172.21.15.72:0/3113461433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34105360 msgr2=0x7f1b34198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b337fe640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34105360 0x7f1b34198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b337fe640 1 -- 172.21.15.72:0/3113461433 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3419fd70 con 0x7f1b34107c90 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b337fe640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34107c90 0x7f1b3419f700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00efd0 tx=0x7f1b2c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b1bfff640 1 -- 172.21.15.72:0/3113461433 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c003b70 con 0x7f1b34107c90 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b341a0060 con 0x7f1b34107c90 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b1bfff640 1 -- 172.21.15.72:0/3113461433 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c003d10 con 0x7f1b34107c90 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b1bfff640 1 -- 172.21.15.72:0/3113461433 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0108a0 con 0x7f1b34107c90 2024-04-25T13:13:04.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.639+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b341a0530 con 0x7f1b34107c90 2024-04-25T13:13:04.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.640+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b340fdf10 con 0x7f1b34107c90 2024-04-25T13:13:04.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.641+0000 7f1b1bfff640 1 -- 172.21.15.72:0/3113461433 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f1b2c004020 con 0x7f1b34107c90 2024-04-25T13:13:04.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.642+0000 7f1b1bfff640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1b1c0a3a60 0x7f1b1c0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:04.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.642+0000 7f1b32ffd640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1b1c0a3a60 0x7f1b1c0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:04.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.642+0000 7f1b1bfff640 1 -- 172.21.15.72:0/3113461433 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f1b2c0d0620 con 0x7f1b34107c90 2024-04-25T13:13:04.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.642+0000 7f1b32ffd640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1b1c0a3a60 0x7f1b1c0a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b24005fd0 tx=0x7f1b24009260 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:04.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.646+0000 7f1b1bfff640 1 -- 172.21.15.72:0/3113461433 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b2c09d1e0 con 0x7f1b34107c90 2024-04-25T13:13:04.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:04.493+0000 7f0f93cab700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:04.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:04.493+0000 7f0f93cab700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:04.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:04.493+0000 7f0f93cab700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:04.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:04.493+0000 7f0f93cab700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:04.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:13:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:13:04.493+0000 7f0f93cab700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:13:04.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.856+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 --> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f1b34062390 con 0x7f1b1c0a3a60 2024-04-25T13:13:04.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.918+0000 7f1b1bfff640 1 -- 172.21.15.72:0/3113461433 <== mgr.15276 v2:172.21.15.72:6800/3170992219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1b34062390 con 0x7f1b1c0a3a60 2024-04-25T13:13:04.919 INFO:teuthology.orchestra.run.smithi072.stdout:Schedule set for path / 2024-04-25T13:13:04.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1b1c0a3a60 msgr2=0x7f1b1c0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:04.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1b1c0a3a60 0x7f1b1c0a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b24005fd0 tx=0x7f1b24009260 comp rx=0 tx=0).stop 2024-04-25T13:13:04.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34107c90 msgr2=0x7f1b3419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:04.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34107c90 0x7f1b3419f700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00efd0 tx=0x7f1b2c00c6a0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 shutdown_connections 2024-04-25T13:13:04.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1b1c0a3a60 0x7f1b1c0a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b34107c90 0x7f1b3419f700 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34105360 0x7f1b34198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 --2- 172.21.15.72:0/3113461433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b34102a30 0x7f1b34198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:04.922 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 >> 172.21.15.72:0/3113461433 conn(0x7f1b340fc420 msgr2=0x7f1b341032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:04.922 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 shutdown_connections 2024-04-25T13:13:04.922 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:04.921+0000 7f1b39405640 1 -- 172.21.15.72:0/3113461433 wait complete. 2024-04-25T13:13:04.932 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T13:13:05.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.035+0000 7f821e835640 1 -- 172.21.15.72:0/2958637259 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218105460 msgr2=0x7f8218107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.035+0000 7f821e835640 1 --2- 172.21.15.72:0/2958637259 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218105460 0x7f8218107850 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f820c009900 tx=0x7f820c02a410 comp rx=0 tx=0).stop 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.035+0000 7f821e835640 1 -- 172.21.15.72:0/2958637259 shutdown_connections 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.035+0000 7f821e835640 1 --2- 172.21.15.72:0/2958637259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8218107d90 0x7f821810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.035+0000 7f821e835640 1 --2- 172.21.15.72:0/2958637259 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218105460 0x7f8218107850 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.035+0000 7f821e835640 1 --2- 172.21.15.72:0/2958637259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8218102b30 0x7f8218104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.035+0000 7f821e835640 1 -- 172.21.15.72:0/2958637259 >> 172.21.15.72:0/2958637259 conn(0x7f82180fc500 msgr2=0x7f82180fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.035+0000 7f821e835640 1 -- 172.21.15.72:0/2958637259 shutdown_connections 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.035+0000 7f821e835640 1 -- 172.21.15.72:0/2958637259 wait complete. 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821e835640 1 Processor -- start 2024-04-25T13:13:05.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821e835640 1 -- start start 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821e835640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8218102b30 0x7f8218198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821e835640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8218105460 0x7f8218198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821e835640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218107d90 0x7f821819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821e835640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f821810db20 con 0x7f8218107d90 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821e835640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f821810d9a0 con 0x7f8218105460 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821e835640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f821810dca0 con 0x7f8218102b30 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821cdd2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218107d90 0x7f821819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821cdd2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218107d90 0x7f821819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37466/0 (socket says 172.21.15.72:37466) 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821cdd2640 1 -- 172.21.15.72:0/4274854045 learned_addr learned my addr 172.21.15.72:0/4274854045 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f82177fe640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8218105460 0x7f8218198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f8217fff640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8218102b30 0x7f8218198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821cdd2640 1 -- 172.21.15.72:0/4274854045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8218102b30 msgr2=0x7f8218198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821cdd2640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8218102b30 0x7f8218198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821cdd2640 1 -- 172.21.15.72:0/4274854045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8218105460 msgr2=0x7f8218198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821cdd2640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8218105460 0x7f8218198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.036+0000 7f821cdd2640 1 -- 172.21.15.72:0/4274854045 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821819fd60 con 0x7f8218107d90 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.037+0000 7f821cdd2640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218107d90 0x7f821819f6f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f821000eaa0 tx=0x7f821000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.037+0000 7f82157fa640 1 -- 172.21.15.72:0/4274854045 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821000cdb0 con 0x7f8218107d90 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.037+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821819fff0 con 0x7f8218107d90 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.037+0000 7f82157fa640 1 -- 172.21.15.72:0/4274854045 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8210005020 con 0x7f8218107d90 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.037+0000 7f82157fa640 1 -- 172.21.15.72:0/4274854045 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82100116d0 con 0x7f8218107d90 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.037+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82181a0510 con 0x7f8218107d90 2024-04-25T13:13:05.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.038+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82180fdff0 con 0x7f8218107d90 2024-04-25T13:13:05.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.039+0000 7f82157fa640 1 -- 172.21.15.72:0/4274854045 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f82100054a0 con 0x7f8218107d90 2024-04-25T13:13:05.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.039+0000 7f82157fa640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f82000a3a60 0x7f82000a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.040+0000 7f82157fa640 1 -- 172.21.15.72:0/4274854045 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f82100d0600 con 0x7f8218107d90 2024-04-25T13:13:05.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.040+0000 7f8217fff640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f82000a3a60 0x7f82000a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.040+0000 7f8217fff640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f82000a3a60 0x7f82000a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8208009bd0 tx=0x7f8208005f90 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:05.043 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.043+0000 7f82157fa640 1 -- 172.21.15.72:0/4274854045 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8210016030 con 0x7f8218107d90 2024-04-25T13:13:05.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.298+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 --> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8218062430 con 0x7f82000a3a60 2024-04-25T13:13:05.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.357+0000 7f82157fa640 1 -- 172.21.15.72:0/4274854045 <== mgr.15276 v2:172.21.15.72:6800/3170992219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8218062430 con 0x7f82000a3a60 2024-04-25T13:13:05.359 INFO:teuthology.orchestra.run.smithi072.stdout:Retention added to path / 2024-04-25T13:13:05.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f82000a3a60 msgr2=0x7f82000a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f82000a3a60 0x7f82000a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8208009bd0 tx=0x7f8208005f90 comp rx=0 tx=0).stop 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218107d90 msgr2=0x7f821819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218107d90 0x7f821819f6f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f821000eaa0 tx=0x7f821000ef60 comp rx=0 tx=0).stop 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 shutdown_connections 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f82000a3a60 0x7f82000a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8218107d90 0x7f821819f6f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8218105460 0x7f8218198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 --2- 172.21.15.72:0/4274854045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8218102b30 0x7f8218198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 >> 172.21.15.72:0/4274854045 conn(0x7f82180fc500 msgr2=0x7f8218103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 shutdown_connections 2024-04-25T13:13:05.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.359+0000 7f821e835640 1 -- 172.21.15.72:0/4274854045 wait complete. 2024-04-25T13:13:05.371 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:13:05.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:05 smithi193 ceph-mon[32743]: [25/Apr/2024:13:13:04] ENGINE Bus STARTING 2024-04-25T13:13:05.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:05 smithi193 ceph-mon[32743]: mgrmap e22: x(active, since 1.01907s), standbys: y, z 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 -- 172.21.15.72:0/3362179736 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c102040 msgr2=0x7f8d0c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 --2- 172.21.15.72:0/3362179736 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c102040 0x7f8d0c1024c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009900 tx=0x7f8cf802a410 comp rx=0 tx=0).stop 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 -- 172.21.15.72:0/3362179736 shutdown_connections 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 --2- 172.21.15.72:0/3362179736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d0c102a00 0x7f8d0c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 --2- 172.21.15.72:0/3362179736 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c102040 0x7f8d0c1024c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 --2- 172.21.15.72:0/3362179736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c100e60 0x7f8d0c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 -- 172.21.15.72:0/3362179736 >> 172.21.15.72:0/3362179736 conn(0x7f8d0c0fc420 msgr2=0x7f8d0c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 -- 172.21.15.72:0/3362179736 shutdown_connections 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 -- 172.21.15.72:0/3362179736 wait complete. 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 Processor -- start 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 -- start start 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c100e60 0x7f8d0c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102040 0x7f8d0c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d0c102a00 0x7f8d0c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c10ca10 con 0x7f8d0c100e60 2024-04-25T13:13:05.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.494+0000 7f8d1324e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c10c890 con 0x7f8d0c102a00 2024-04-25T13:13:05.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d1324e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c10cb90 con 0x7f8d0c102040 2024-04-25T13:13:05.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d10fea640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c100e60 0x7f8d0c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d10fea640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c100e60 0x7f8d0c06a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34252/0 (socket says 172.21.15.72:34252) 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d10fea640 1 -- 172.21.15.72:0/1656690914 learned_addr learned my addr 172.21.15.72:0/1656690914 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d0bfff640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102040 0x7f8d0c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d117eb640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d0c102a00 0x7f8d0c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d10fea640 1 -- 172.21.15.72:0/1656690914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102040 msgr2=0x7f8d0c19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d10fea640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102040 0x7f8d0c19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d10fea640 1 -- 172.21.15.72:0/1656690914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d0c102a00 msgr2=0x7f8d0c1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d10fea640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d0c102a00 0x7f8d0c1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d10fea640 1 -- 172.21.15.72:0/1656690914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c1a1ff0 con 0x7f8d0c100e60 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d10fea640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c100e60 0x7f8d0c06a170 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000e9c0 tx=0x7f8d0000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d09ffb640 1 -- 172.21.15.72:0/1656690914 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0000cd80 con 0x7f8d0c100e60 2024-04-25T13:13:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c1a2280 con 0x7f8d0c100e60 2024-04-25T13:13:05.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d09ffb640 1 -- 172.21.15.72:0/1656690914 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d000035f0 con 0x7f8d0c100e60 2024-04-25T13:13:05.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.495+0000 7f8d09ffb640 1 -- 172.21.15.72:0/1656690914 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00010690 con 0x7f8d0c100e60 2024-04-25T13:13:05.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.496+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c1a27a0 con 0x7f8d0c100e60 2024-04-25T13:13:05.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.497+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c108280 con 0x7f8d0c100e60 2024-04-25T13:13:05.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.498+0000 7f8d09ffb640 1 -- 172.21.15.72:0/1656690914 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f8d00004000 con 0x7f8d0c100e60 2024-04-25T13:13:05.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.498+0000 7f8d09ffb640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f8cfc0a3a90 0x7f8cfc0a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.499+0000 7f8d0bfff640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f8cfc0a3a90 0x7f8cfc0a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.499+0000 7f8d09ffb640 1 -- 172.21.15.72:0/1656690914 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f8d000d0760 con 0x7f8d0c100e60 2024-04-25T13:13:05.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.499+0000 7f8d0bfff640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f8cfc0a3a90 0x7f8cfc0a5f50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8000c00 tx=0x7f8cf80314a0 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:05.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.503+0000 7f8d09ffb640 1 -- 172.21.15.72:0/1656690914 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d0009d320 con 0x7f8d0c100e60 2024-04-25T13:13:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:05 smithi072 ceph-mon[29199]: [25/Apr/2024:13:13:04] ENGINE Bus STARTING 2024-04-25T13:13:05.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:05 smithi072 ceph-mon[29199]: mgrmap e22: x(active, since 1.01907s), standbys: y, z 2024-04-25T13:13:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:05 smithi164 ceph-mon[32418]: [25/Apr/2024:13:13:04] ENGINE Bus STARTING 2024-04-25T13:13:05.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:05 smithi164 ceph-mon[32418]: mgrmap e22: x(active, since 1.01907s), standbys: y, z 2024-04-25T13:13:05.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.728+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 --> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f8d0c062390 con 0x7f8cfc0a3a90 2024-04-25T13:13:05.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.736+0000 7f8d09ffb640 1 -- 172.21.15.72:0/1656690914 <== mgr.15276 v2:172.21.15.72:6800/3170992219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f8d0c062390 con 0x7f8cfc0a3a90 2024-04-25T13:13:05.736 INFO:teuthology.orchestra.run.smithi072.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:13:04", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T13:13:05.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f8cfc0a3a90 msgr2=0x7f8cfc0a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f8cfc0a3a90 0x7f8cfc0a5f50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8000c00 tx=0x7f8cf80314a0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c100e60 msgr2=0x7f8d0c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c100e60 0x7f8d0c06a170 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000e9c0 tx=0x7f8d0000ee80 comp rx=0 tx=0).stop 2024-04-25T13:13:05.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 shutdown_connections 2024-04-25T13:13:05.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f8cfc0a3a90 0x7f8cfc0a5f50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d0c102a00 0x7f8d0c1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102040 0x7f8d0c19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 --2- 172.21.15.72:0/1656690914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d0c100e60 0x7f8d0c06a170 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 >> 172.21.15.72:0/1656690914 conn(0x7f8d0c0fc420 msgr2=0x7f8d0c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:05.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 shutdown_connections 2024-04-25T13:13:05.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.738+0000 7f8d1324e640 1 -- 172.21.15.72:0/1656690914 wait complete. 2024-04-25T13:13:05.749 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:13:05.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.851+0000 7f4442d4a640 1 -- 172.21.15.72:0/522146125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 msgr2=0x7f443c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.851+0000 7f4442d4a640 1 --2- 172.21.15.72:0/522146125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 0x7f443c101280 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4430009900 tx=0x7f443002a410 comp rx=0 tx=0).stop 2024-04-25T13:13:05.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.851+0000 7f4442d4a640 1 -- 172.21.15.72:0/522146125 shutdown_connections 2024-04-25T13:13:05.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.851+0000 7f4442d4a640 1 --2- 172.21.15.72:0/522146125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443c102a00 0x7f443c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.851+0000 7f4442d4a640 1 --2- 172.21.15.72:0/522146125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c102040 0x7f443c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.851+0000 7f4442d4a640 1 --2- 172.21.15.72:0/522146125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 0x7f443c101280 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.851+0000 7f4442d4a640 1 -- 172.21.15.72:0/522146125 >> 172.21.15.72:0/522146125 conn(0x7f443c0fc420 msgr2=0x7f443c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.851+0000 7f4442d4a640 1 -- 172.21.15.72:0/522146125 shutdown_connections 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.851+0000 7f4442d4a640 1 -- 172.21.15.72:0/522146125 wait complete. 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4442d4a640 1 Processor -- start 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4442d4a640 1 -- start start 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4442d4a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 0x7f443c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4442d4a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443c102040 0x7f443c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4442d4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c102a00 0x7f443c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4442d4a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f443c10c9f0 con 0x7f443c100e60 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4442d4a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f443c10c870 con 0x7f443c102a00 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4442d4a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f443c10cb70 con 0x7f443c102040 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4440ae6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 0x7f443c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4440ae6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 0x7f443c198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34298/0 (socket says 172.21.15.72:34298) 2024-04-25T13:13:05.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4440ae6640 1 -- 172.21.15.72:0/934238253 learned_addr learned my addr 172.21.15.72:0/934238253 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:13:05.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f44412e7640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c102a00 0x7f443c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f443bfff640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443c102040 0x7f443c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4440ae6640 1 -- 172.21.15.72:0/934238253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443c102040 msgr2=0x7f443c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4440ae6640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443c102040 0x7f443c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4440ae6640 1 -- 172.21.15.72:0/934238253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c102a00 msgr2=0x7f443c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:05.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4440ae6640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c102a00 0x7f443c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:05.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4440ae6640 1 -- 172.21.15.72:0/934238253 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c19fdc0 con 0x7f443c100e60 2024-04-25T13:13:05.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4440ae6640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 0x7f443c198210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4430006010 tx=0x7f44300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:05.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.852+0000 7f4439ffb640 1 -- 172.21.15.72:0/934238253 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443003f030 con 0x7f443c100e60 2024-04-25T13:13:05.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.853+0000 7f4439ffb640 1 -- 172.21.15.72:0/934238253 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4430031910 con 0x7f443c100e60 2024-04-25T13:13:05.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.853+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c1a0050 con 0x7f443c100e60 2024-04-25T13:13:05.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.853+0000 7f4439ffb640 1 -- 172.21.15.72:0/934238253 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430035070 con 0x7f443c100e60 2024-04-25T13:13:05.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.853+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c1a0540 con 0x7f443c100e60 2024-04-25T13:13:05.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.854+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c1a07a0 con 0x7f443c100e60 2024-04-25T13:13:05.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.855+0000 7f4439ffb640 1 -- 172.21.15.72:0/934238253 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f4430042030 con 0x7f443c100e60 2024-04-25T13:13:05.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.855+0000 7f4439ffb640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f442c0a3a60 0x7f442c0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:05.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.856+0000 7f443bfff640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f442c0a3a60 0x7f442c0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:05.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.856+0000 7f4439ffb640 1 -- 172.21.15.72:0/934238253 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f44300f0600 con 0x7f443c100e60 2024-04-25T13:13:05.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.856+0000 7f443bfff640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f442c0a3a60 0x7f442c0a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f443c1996b0 tx=0x7f4428005f90 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:05.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:05.860+0000 7f4439ffb640 1 -- 172.21.15.72:0/934238253 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4430003e60 con 0x7f443c100e60 2024-04-25T13:13:06.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.077+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 --> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f443c0fe600 con 0x7f442c0a3a60 2024-04-25T13:13:06.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.089+0000 7f4439ffb640 1 -- 172.21.15.72:0/934238253 <== mgr.15276 v2:172.21.15.72:6800/3170992219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f443c0fe600 con 0x7f442c0a3a60 2024-04-25T13:13:06.089 INFO:teuthology.orchestra.run.smithi072.stdout:/ 1m 6m3h 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.091+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f442c0a3a60 msgr2=0x7f442c0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.091+0000 7f4442d4a640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f442c0a3a60 0x7f442c0a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f443c1996b0 tx=0x7f4428005f90 comp rx=0 tx=0).stop 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.091+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 msgr2=0x7f443c198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.091+0000 7f4442d4a640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 0x7f443c198210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4430006010 tx=0x7f44300042f0 comp rx=0 tx=0).stop 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.092+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 shutdown_connections 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.092+0000 7f4442d4a640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f442c0a3a60 0x7f442c0a5f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.092+0000 7f4442d4a640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c102a00 0x7f443c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.092+0000 7f4442d4a640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443c102040 0x7f443c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.092+0000 7f4442d4a640 1 --2- 172.21.15.72:0/934238253 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f443c100e60 0x7f443c198210 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.092+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 >> 172.21.15.72:0/934238253 conn(0x7f443c0fc420 msgr2=0x7f443c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.092+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 shutdown_connections 2024-04-25T13:13:06.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:13:06.092+0000 7f4442d4a640 1 -- 172.21.15.72:0/934238253 wait complete. 2024-04-25T13:13:06.104 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T13:13:06.129 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T13:13:06.139 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T13:13:06.199 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:13:06.199 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T13:13:06.201 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T13:13:06.214 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:13:06.466 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=e6b72320fc8240599e9357b283b792e8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:13:06.468 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T13:13:06.495 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T13:13:06.514 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:13:06.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: [25/Apr/2024:13:13:04] ENGINE Serving on https://172.21.15.72:7150 2024-04-25T13:13:06.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:06.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:13:06.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: [25/Apr/2024:13:13:04] ENGINE Serving on http://172.21.15.72:8765 2024-04-25T13:13:06.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: [25/Apr/2024:13:13:04] ENGINE Bus STARTED 2024-04-25T13:13:06.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:06.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:06.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:06 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: [25/Apr/2024:13:13:04] ENGINE Serving on https://172.21.15.72:7150 2024-04-25T13:13:06.528 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:06.528 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:13:06.528 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: [25/Apr/2024:13:13:04] ENGINE Serving on http://172.21.15.72:8765 2024-04-25T13:13:06.528 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: [25/Apr/2024:13:13:04] ENGINE Bus STARTED 2024-04-25T13:13:06.528 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:06.528 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:06.528 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.528 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.529 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.529 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.529 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.529 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:06 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.531 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T13:13:06.541 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T13:13:06.541 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:13:06.541 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:13:06.541 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T13:13:06.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: [25/Apr/2024:13:13:04] ENGINE Serving on https://172.21.15.72:7150 2024-04-25T13:13:06.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:06.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:13:06.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: [25/Apr/2024:13:13:04] ENGINE Serving on http://172.21.15.72:8765 2024-04-25T13:13:06.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: [25/Apr/2024:13:13:04] ENGINE Bus STARTED 2024-04-25T13:13:06.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:06.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:06.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:06 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:06.596 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T13:13:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:13:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:07.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:07.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T13:13:07.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:07.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:07 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:13:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T13:13:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:07.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:07 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:07.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:13:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T13:13:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:07.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:07.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:07.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:07 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T13:13:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T13:13:08.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-04-25T13:13:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-25T13:13:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-04-25T13:13:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Unable to set osd_memory_target on smithi072 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-25T13:13:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-25T13:13:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T13:13:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T13:13:08.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:13:08.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:13:08.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:13:08.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:08.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:08.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:08.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:08.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:08.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:08.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T13:13:08.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T13:13:08.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-04-25T13:13:08.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-25T13:13:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-04-25T13:13:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Unable to set osd_memory_target on smithi072 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-25T13:13:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-25T13:13:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T13:13:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T13:13:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:13:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:13:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:13:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:08.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:08.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:08.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.526 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:08.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:08.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:08.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T13:13:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T13:13:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-04-25T13:13:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-25T13:13:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-04-25T13:13:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Unable to set osd_memory_target on smithi072 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-25T13:13:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-25T13:13:08.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T13:13:08.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T13:13:08.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:13:08.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:13:08.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:13:08.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:08.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:08.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:08.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:08.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:08.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:08.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:08.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:13:09.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:09 smithi072 ceph-mon[29199]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:13:09.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:09 smithi072 ceph-mon[29199]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:13:09.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:09 smithi072 ceph-mon[29199]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:13:09.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:09 smithi072 ceph-mon[29199]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:13:09.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:09 smithi072 ceph-mon[29199]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T13:13:09.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:09 smithi164 ceph-mon[32418]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:13:09.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:09 smithi164 ceph-mon[32418]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:13:09.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:09 smithi164 ceph-mon[32418]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:13:09.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:09 smithi164 ceph-mon[32418]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:13:09.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:09 smithi164 ceph-mon[32418]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T13:13:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:09 smithi193 ceph-mon[32743]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:13:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:09 smithi193 ceph-mon[32743]: Updating smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:13:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:09 smithi193 ceph-mon[32743]: Updating smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:13:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:09 smithi193 ceph-mon[32743]: Updating smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:13:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:09 smithi193 ceph-mon[32743]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T13:13:11.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:11 smithi072 ceph-mon[29199]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T13:13:11.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:11 smithi164 ceph-mon[32418]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T13:13:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:11 smithi193 ceph-mon[32743]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T13:13:13.527 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:13 smithi072 ceph-mon[29199]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-25T13:13:13.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:13 smithi164 ceph-mon[32418]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-25T13:13:13.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:13 smithi193 ceph-mon[32743]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-25T13:13:15.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:15 smithi072 ceph-mon[29199]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 60 op/s 2024-04-25T13:13:15.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:15 smithi164 ceph-mon[32418]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 60 op/s 2024-04-25T13:13:15.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:15 smithi193 ceph-mon[32743]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 60 op/s 2024-04-25T13:13:17.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:17 smithi072 ceph-mon[29199]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-25T13:13:17.533 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:17 smithi193 ceph-mon[32743]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-25T13:13:17.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:17 smithi164 ceph-mon[32418]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-25T13:13:19.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:19 smithi072 ceph-mon[29199]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T13:13:19.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:19 smithi164 ceph-mon[32418]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T13:13:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:19 smithi193 ceph-mon[32743]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T13:13:21.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:21 smithi072 ceph-mon[29199]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-25T13:13:21.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:21 smithi164 ceph-mon[32418]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-25T13:13:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:21 smithi193 ceph-mon[32743]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-25T13:13:23.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:23 smithi072 ceph-mon[29199]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-25T13:13:23.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:23 smithi164 ceph-mon[32418]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-25T13:13:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:23 smithi193 ceph-mon[32743]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-25T13:13:25.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:25 smithi072 ceph-mon[29199]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:25.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:25 smithi164 ceph-mon[32418]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:25 smithi193 ceph-mon[32743]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:27.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:27 smithi072 ceph-mon[29199]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:27.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:27 smithi164 ceph-mon[32418]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:27 smithi193 ceph-mon[32743]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:29.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:29 smithi072 ceph-mon[29199]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T13:13:29.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:29 smithi164 ceph-mon[32418]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T13:13:29.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:29 smithi193 ceph-mon[32743]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T13:13:31.523 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:31 smithi072 ceph-mon[29199]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T13:13:31.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:31 smithi164 ceph-mon[32418]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T13:13:31.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:31 smithi193 ceph-mon[32743]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T13:13:33.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:33 smithi072 ceph-mon[29199]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T13:13:33.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:33 smithi164 ceph-mon[32418]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T13:13:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:33 smithi193 ceph-mon[32743]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T13:13:35.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:35 smithi072 ceph-mon[29199]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T13:13:35.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:35 smithi164 ceph-mon[32418]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T13:13:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:35 smithi193 ceph-mon[32743]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T13:13:37.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:37 smithi164 ceph-mon[32418]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-25T13:13:37.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:37 smithi193 ceph-mon[32743]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-25T13:13:37.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:37 smithi072 ceph-mon[29199]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-25T13:13:39.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:39 smithi164 ceph-mon[32418]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T13:13:39.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:39 smithi193 ceph-mon[32743]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T13:13:39.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:39 smithi072 ceph-mon[29199]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T13:13:41.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:41 smithi164 ceph-mon[32418]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T13:13:41.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:41 smithi193 ceph-mon[32743]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T13:13:41.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:41 smithi072 ceph-mon[29199]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T13:13:43.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:43 smithi164 ceph-mon[32418]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T13:13:43.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:43 smithi193 ceph-mon[32743]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T13:13:43.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:43 smithi072 ceph-mon[29199]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T13:13:45.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:45 smithi164 ceph-mon[32418]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:45.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:45 smithi193 ceph-mon[32743]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:45.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:45 smithi072 ceph-mon[29199]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:47.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:47 smithi164 ceph-mon[32418]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:47.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:47 smithi193 ceph-mon[32743]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:47.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:47 smithi072 ceph-mon[29199]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:49.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:49 smithi164 ceph-mon[32418]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:49.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:49 smithi072 ceph-mon[29199]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:49.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:49 smithi193 ceph-mon[32743]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:51.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:51 smithi164 ceph-mon[32418]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:51.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:51 smithi193 ceph-mon[32743]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:51.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:51 smithi072 ceph-mon[29199]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:53.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:53 smithi164 ceph-mon[32418]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:13:53.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:53 smithi193 ceph-mon[32743]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:13:53.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:53 smithi072 ceph-mon[29199]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:13:55.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:55 smithi164 ceph-mon[32418]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:55.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:55 smithi193 ceph-mon[32743]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:55.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:55 smithi072 ceph-mon[29199]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:57.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:57 smithi164 ceph-mon[32418]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:57.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:57 smithi193 ceph-mon[32743]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:57.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:57 smithi072 ceph-mon[29199]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:59.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:13:59 smithi164 ceph-mon[32418]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:59.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:13:59 smithi193 ceph-mon[32743]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:59.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:13:59 smithi072 ceph-mon[29199]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:00.774 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:14:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:14:00.367+0000 7f0f7302a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:14:00.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:14:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:14:00.367+0000 7f0f7302a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:14:00.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:14:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:14:00.367+0000 7f0f7302a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:14:00.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:14:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:14:00.367+0000 7f0f7302a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:14:00.775 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:14:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x[29432]: 2024-04-25T13:14:00.367+0000 7f0f7302a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:14:01.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:01 smithi193 ceph-mon[32743]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:14:01.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:01 smithi072 ceph-mon[29199]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:14:01.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:01 smithi164 ceph-mon[32418]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:14:03.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:03 smithi193 ceph-mon[32743]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:03.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:03 smithi193 ceph-mon[32743]: mgrmap e25: x(active, since 58s), standbys: y, z 2024-04-25T13:14:03.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:03 smithi072 ceph-mon[29199]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:03.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:03 smithi072 ceph-mon[29199]: mgrmap e25: x(active, since 58s), standbys: y, z 2024-04-25T13:14:03.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:03 smithi164 ceph-mon[32418]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:03.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:03 smithi164 ceph-mon[32418]: mgrmap e25: x(active, since 58s), standbys: y, z 2024-04-25T13:14:05.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:05 smithi193 ceph-mon[32743]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:05.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:05 smithi072 ceph-mon[29199]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:05.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:05 smithi164 ceph-mon[32418]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:07 smithi193 ceph-mon[32743]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:14:07.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:07 smithi072 ceph-mon[29199]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:14:07.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:07 smithi164 ceph-mon[32418]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:14:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:08 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:08.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:08 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:08.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:08 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:09 smithi193 ceph-mon[32743]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:14:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:09 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:14:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:09 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:14:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:09 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:09.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:09 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:09.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:09 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:14:09.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:09 smithi072 ceph-mon[29199]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:14:09.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:09 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:14:09.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:09 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:14:09.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:09 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:09.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:09 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:09.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:09 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:14:09.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:09 smithi164 ceph-mon[32418]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:14:09.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:09 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:14:09.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:09 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:14:09.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:09 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:09.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:09 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:09.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:09 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:14:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:11 smithi193 ceph-mon[32743]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:14:11.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:11 smithi072 ceph-mon[29199]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:14:11.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:11 smithi164 ceph-mon[32418]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:14:13.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:13 smithi193 ceph-mon[32743]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:14:13.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:13 smithi072 ceph-mon[29199]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:14:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:13 smithi164 ceph-mon[32418]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:14:15.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:15 smithi193 ceph-mon[32743]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:14:15.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:15 smithi072 ceph-mon[29199]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:14:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:15 smithi164 ceph-mon[32418]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:14:17.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:17 smithi193 ceph-mon[32743]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:14:17.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:17 smithi072 ceph-mon[29199]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:14:17.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:17 smithi164 ceph-mon[32418]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:14:19.609 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:19 smithi164 ceph-mon[32418]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:14:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:19 smithi193 ceph-mon[32743]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:14:19.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:19 smithi072 ceph-mon[29199]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:14:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:21 smithi193 ceph-mon[32743]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:14:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:21 smithi072 ceph-mon[29199]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:14:21.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:21 smithi164 ceph-mon[32418]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:14:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:23 smithi193 ceph-mon[32743]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:14:23.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:23 smithi072 ceph-mon[29199]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:14:23.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:23 smithi164 ceph-mon[32418]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:14:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:25 smithi193 ceph-mon[32743]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:14:25.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:25 smithi072 ceph-mon[29199]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:14:25.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:25 smithi164 ceph-mon[32418]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:14:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:27 smithi193 ceph-mon[32743]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T13:14:27.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:27 smithi072 ceph-mon[29199]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T13:14:27.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:27 smithi164 ceph-mon[32418]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T13:14:29.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:29 smithi193 ceph-mon[32743]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:29.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:29 smithi072 ceph-mon[29199]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:29.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:29 smithi164 ceph-mon[32418]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:31.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:31 smithi193 ceph-mon[32743]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:14:31.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:31 smithi072 ceph-mon[29199]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:14:31.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:31 smithi164 ceph-mon[32418]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:14:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:33 smithi193 ceph-mon[32743]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:33.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:33 smithi072 ceph-mon[29199]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:33 smithi164 ceph-mon[32418]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:35 smithi193 ceph-mon[32743]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:35.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:35 smithi072 ceph-mon[29199]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:35 smithi164 ceph-mon[32418]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:36.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:36 smithi193 ceph-mon[32743]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-25T13:14:36.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:36 smithi193 ceph-mon[32743]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T13:14:36.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:36 smithi072 ceph-mon[29199]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-25T13:14:36.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:36 smithi072 ceph-mon[29199]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T13:14:36.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:36 smithi164 ceph-mon[32418]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-25T13:14:36.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:36 smithi164 ceph-mon[32418]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T13:14:38.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:38 smithi193 ceph-mon[32743]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-25T13:14:39.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:38 smithi072 ceph-mon[29199]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-25T13:14:39.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:38 smithi164 ceph-mon[32418]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-25T13:14:40.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:40 smithi193 ceph-mon[32743]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-25T13:14:41.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:40 smithi072 ceph-mon[29199]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-25T13:14:41.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:40 smithi164 ceph-mon[32418]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-25T13:14:42.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:42 smithi193 ceph-mon[32743]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-25T13:14:43.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:42 smithi072 ceph-mon[29199]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-25T13:14:43.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:42 smithi164 ceph-mon[32418]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-25T13:14:44.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:44 smithi193 ceph-mon[32743]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-25T13:14:45.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:44 smithi072 ceph-mon[29199]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-25T13:14:45.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:44 smithi164 ceph-mon[32418]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-25T13:14:46.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:46 smithi193 ceph-mon[32743]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-25T13:14:47.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:46 smithi072 ceph-mon[29199]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-25T13:14:47.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:46 smithi164 ceph-mon[32418]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-25T13:14:48.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:48 smithi193 ceph-mon[32743]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T13:14:49.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:48 smithi072 ceph-mon[29199]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T13:14:49.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:48 smithi164 ceph-mon[32418]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T13:14:50.859 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:50 smithi164 ceph-mon[32418]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T13:14:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:50 smithi193 ceph-mon[32743]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T13:14:51.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:50 smithi072 ceph-mon[29199]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T13:14:52.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:52 smithi193 ceph-mon[32743]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:53.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:52 smithi072 ceph-mon[29199]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:53.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:52 smithi164 ceph-mon[32418]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:54 smithi193 ceph-mon[32743]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:55.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:54 smithi072 ceph-mon[29199]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:55.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:54 smithi164 ceph-mon[32418]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:56.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:56 smithi193 ceph-mon[32743]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:14:57.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:56 smithi072 ceph-mon[29199]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:14:57.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:56 smithi164 ceph-mon[32418]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:14:58.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:14:58 smithi193 ceph-mon[32743]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:59.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:14:58 smithi072 ceph-mon[29199]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:14:59.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:14:58 smithi164 ceph-mon[32418]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:00.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:00 smithi193 ceph-mon[32743]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:01.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:00 smithi072 ceph-mon[29199]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:01.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:00 smithi164 ceph-mon[32418]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:03.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:03 smithi193 ceph-mon[32743]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:03.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:03 smithi072 ceph-mon[29199]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:03.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:03 smithi164 ceph-mon[32418]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:05.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:05 smithi193 ceph-mon[32743]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:05.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:05 smithi072 ceph-mon[29199]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:05.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:05 smithi164 ceph-mon[32418]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:07 smithi193 ceph-mon[32743]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:15:07.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:07 smithi072 ceph-mon[29199]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:15:07.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:07 smithi164 ceph-mon[32418]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:15:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:09 smithi193 ceph-mon[32743]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:15:09.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:09 smithi072 ceph-mon[29199]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:15:09.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:09 smithi164 ceph-mon[32418]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:15:10.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:10 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:10.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:10 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:10 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:10 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:10 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:10 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:10.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:10 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:10 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:10 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:10 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:10 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:10.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:10 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:10 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:10 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:10.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:10 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:11 smithi193 ceph-mon[32743]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:15:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:11 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:11 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:11 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:11.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:11 smithi072 ceph-mon[29199]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:15:11.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:11 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:11.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:11 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:11.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:11 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:11.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:11 smithi164 ceph-mon[32418]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:15:11.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:11 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:11.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:11 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:11.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:11 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:15:13.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:13 smithi193 ceph-mon[32743]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:15:13.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:13 smithi072 ceph-mon[29199]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:15:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:13 smithi164 ceph-mon[32418]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:15:15.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:15 smithi193 ceph-mon[32743]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:15:15.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:15 smithi072 ceph-mon[29199]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:15:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:15 smithi164 ceph-mon[32418]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:15:17.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:17 smithi193 ceph-mon[32743]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:15:17.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:17 smithi072 ceph-mon[29199]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:15:17.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:17 smithi164 ceph-mon[32418]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:15:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:19 smithi193 ceph-mon[32743]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:15:19.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:19 smithi072 ceph-mon[29199]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:15:19.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:19 smithi164 ceph-mon[32418]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:15:21.609 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:21 smithi164 ceph-mon[32418]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:15:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:21 smithi193 ceph-mon[32743]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:15:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:21 smithi072 ceph-mon[29199]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:15:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:23 smithi193 ceph-mon[32743]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:15:23.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:23 smithi072 ceph-mon[29199]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:15:23.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:23 smithi164 ceph-mon[32418]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:15:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:25 smithi193 ceph-mon[32743]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:15:25.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:25 smithi072 ceph-mon[29199]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:15:25.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:25 smithi164 ceph-mon[32418]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:15:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:27 smithi193 ceph-mon[32743]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T13:15:27.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:27 smithi072 ceph-mon[29199]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T13:15:27.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:27 smithi164 ceph-mon[32418]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T13:15:29.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:29 smithi193 ceph-mon[32743]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:29.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:29 smithi072 ceph-mon[29199]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:29.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:29 smithi164 ceph-mon[32418]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:31.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:31 smithi193 ceph-mon[32743]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:31.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:31 smithi072 ceph-mon[29199]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:31.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:31 smithi164 ceph-mon[32418]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:33 smithi193 ceph-mon[32743]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:33.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:33 smithi072 ceph-mon[29199]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:33 smithi164 ceph-mon[32418]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:35 smithi193 ceph-mon[32743]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:35.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:35 smithi072 ceph-mon[29199]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:35 smithi164 ceph-mon[32418]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:37.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:37 smithi193 ceph-mon[32743]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:37.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:37 smithi072 ceph-mon[29199]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:37.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:37 smithi164 ceph-mon[32418]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:39.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:39 smithi193 ceph-mon[32743]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:39.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:39 smithi072 ceph-mon[29199]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:39.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:39 smithi164 ceph-mon[32418]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:41.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:41 smithi193 ceph-mon[32743]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:41.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:41 smithi072 ceph-mon[29199]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:41 smithi164 ceph-mon[32418]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:43.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:43 smithi193 ceph-mon[32743]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:43.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:43 smithi072 ceph-mon[29199]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:43.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:43 smithi164 ceph-mon[32418]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:45.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:45 smithi193 ceph-mon[32743]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:45.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:45 smithi072 ceph-mon[29199]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:45.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:45 smithi164 ceph-mon[32418]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:47.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:47 smithi193 ceph-mon[32743]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:47.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:47 smithi072 ceph-mon[29199]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:47.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:47 smithi164 ceph-mon[32418]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:48.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:48 smithi193 ceph-mon[32743]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:48.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:48 smithi072 ceph-mon[29199]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:48 smithi164 ceph-mon[32418]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:50 smithi193 ceph-mon[32743]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:51.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:50 smithi072 ceph-mon[29199]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:51.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:50 smithi164 ceph-mon[32418]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:52.859 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:52 smithi164 ceph-mon[32418]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:52.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:52 smithi193 ceph-mon[32743]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:53.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:52 smithi072 ceph-mon[29199]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:54 smithi193 ceph-mon[32743]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:55.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:54 smithi072 ceph-mon[29199]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:55.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:54 smithi164 ceph-mon[32418]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:56.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:56 smithi193 ceph-mon[32743]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:57.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:56 smithi072 ceph-mon[29199]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:57.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:56 smithi164 ceph-mon[32418]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:15:58.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:15:58 smithi193 ceph-mon[32743]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:59.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:15:58 smithi072 ceph-mon[29199]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:15:59.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:15:58 smithi164 ceph-mon[32418]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:00.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:00 smithi193 ceph-mon[32743]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:01.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:00 smithi072 ceph-mon[29199]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:01.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:00 smithi164 ceph-mon[32418]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:02.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:02 smithi193 ceph-mon[32743]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:03.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:02 smithi072 ceph-mon[29199]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:03.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:02 smithi164 ceph-mon[32418]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:04.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:04 smithi193 ceph-mon[32743]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:05.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:04 smithi072 ceph-mon[29199]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:05.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:04 smithi164 ceph-mon[32418]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:06.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:06 smithi193 ceph-mon[32743]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:16:07.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:06 smithi072 ceph-mon[29199]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:16:07.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:06 smithi164 ceph-mon[32418]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:16:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:08 smithi193 ceph-mon[32743]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:16:09.027 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:08 smithi072 ceph-mon[29199]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:16:09.060 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:08 smithi164 ceph-mon[32418]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:16:10.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:10 smithi193 ceph-mon[32743]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:16:11.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:10 smithi072 ceph-mon[29199]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:16:11.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:10 smithi164 ceph-mon[32418]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:16:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:11 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:11 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:11 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:11.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:11 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:16:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:11 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:11 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:11 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:11 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:16:12.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:11 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:12.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:11 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:12.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:11 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:12.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:11 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:16:12.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:12 smithi193 ceph-mon[32743]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:16:13.023 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:12 smithi072 ceph-mon[29199]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:16:13.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:12 smithi164 ceph-mon[32418]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:16:14.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:14 smithi193 ceph-mon[32743]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:16:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:14 smithi072 ceph-mon[29199]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:16:15.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:14 smithi164 ceph-mon[32418]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:16:16.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:16 smithi193 ceph-mon[32743]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:16:17.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:16 smithi072 ceph-mon[29199]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:16:17.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:16 smithi164 ceph-mon[32418]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:16:18.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:18 smithi193 ceph-mon[32743]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:16:19.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:18 smithi072 ceph-mon[29199]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:16:19.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:18 smithi164 ceph-mon[32418]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:16:20.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:20 smithi193 ceph-mon[32743]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:16:21.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:20 smithi072 ceph-mon[29199]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:16:21.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:20 smithi164 ceph-mon[32418]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:16:22.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:22 smithi193 ceph-mon[32743]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:23.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:22 smithi072 ceph-mon[29199]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:23.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:22 smithi164 ceph-mon[32418]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:24.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:24 smithi193 ceph-mon[32743]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:25.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:24 smithi072 ceph-mon[29199]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:25.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:24 smithi164 ceph-mon[32418]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:26.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:26 smithi193 ceph-mon[32743]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:27.026 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:26 smithi072 ceph-mon[29199]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:27.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:26 smithi164 ceph-mon[32418]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:28.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:28 smithi193 ceph-mon[32743]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:29.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:28 smithi072 ceph-mon[29199]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:29.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:28 smithi164 ceph-mon[32418]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:30.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:30 smithi193 ceph-mon[32743]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:31.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:30 smithi072 ceph-mon[29199]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:31.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:30 smithi164 ceph-mon[32418]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:32.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:32 smithi193 ceph-mon[32743]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:33.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:32 smithi072 ceph-mon[29199]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:33.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:32 smithi164 ceph-mon[32418]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:34.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:34 smithi193 ceph-mon[32743]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:35.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:34 smithi072 ceph-mon[29199]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:35.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:34 smithi164 ceph-mon[32418]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:36.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:36 smithi193 ceph-mon[32743]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:37.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:36 smithi072 ceph-mon[29199]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:37.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:36 smithi164 ceph-mon[32418]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:38.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:38 smithi193 ceph-mon[32743]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:39.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:38 smithi072 ceph-mon[29199]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:39.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:38 smithi164 ceph-mon[32418]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:40.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:40 smithi193 ceph-mon[32743]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:41.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:40 smithi072 ceph-mon[29199]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:41.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:40 smithi164 ceph-mon[32418]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:42.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:42 smithi193 ceph-mon[32743]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:43.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:42 smithi072 ceph-mon[29199]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:43.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:42 smithi164 ceph-mon[32418]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:44.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:44 smithi193 ceph-mon[32743]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:45.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:44 smithi072 ceph-mon[29199]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:45.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:44 smithi164 ceph-mon[32418]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:46.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:46 smithi193 ceph-mon[32743]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:47.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:46 smithi072 ceph-mon[29199]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:47.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:46 smithi164 ceph-mon[32418]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:48.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:48 smithi193 ceph-mon[32743]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:49.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:48 smithi072 ceph-mon[29199]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:49.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:48 smithi164 ceph-mon[32418]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:50 smithi193 ceph-mon[32743]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:51.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:50 smithi072 ceph-mon[29199]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:51.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:50 smithi164 ceph-mon[32418]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:52.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:52 smithi193 ceph-mon[32743]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:53.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:52 smithi072 ceph-mon[29199]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:53.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:52 smithi164 ceph-mon[32418]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:54.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:54 smithi193 ceph-mon[32743]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:55.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:54 smithi072 ceph-mon[29199]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:55.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:54 smithi164 ceph-mon[32418]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:56.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:56 smithi193 ceph-mon[32743]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:57.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:56 smithi072 ceph-mon[29199]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:57.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:56 smithi164 ceph-mon[32418]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:16:58.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:16:58 smithi193 ceph-mon[32743]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:59.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:16:58 smithi072 ceph-mon[29199]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:16:59.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:16:58 smithi164 ceph-mon[32418]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:00.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:00 smithi193 ceph-mon[32743]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:01.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:00 smithi072 ceph-mon[29199]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:01.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:00 smithi164 ceph-mon[32418]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:02.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:02 smithi193 ceph-mon[32743]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:03.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:02 smithi072 ceph-mon[29199]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:03.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:02 smithi164 ceph-mon[32418]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:04.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:04 smithi193 ceph-mon[32743]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:05.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:04 smithi072 ceph-mon[29199]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:05.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:04 smithi164 ceph-mon[32418]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:06.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:06 smithi193 ceph-mon[32743]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:17:07.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:06 smithi072 ceph-mon[29199]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:17:07.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:06 smithi164 ceph-mon[32418]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:17:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:08 smithi193 ceph-mon[32743]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:17:09.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:08 smithi072 ceph-mon[29199]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:17:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:08 smithi164 ceph-mon[32418]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:17:10.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:10 smithi193 ceph-mon[32743]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:17:11.026 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:10 smithi072 ceph-mon[29199]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:17:11.062 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:10 smithi164 ceph-mon[32418]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:17:11.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:11 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:12.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:11 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:12.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:11 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:12.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:12 smithi193 ceph-mon[32743]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:17:12.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:12 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:12.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:12 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:12.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:12 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:17:13.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:12 smithi072 ceph-mon[29199]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:17:13.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:12 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:13.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:12 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:13.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:12 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:17:13.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:12 smithi164 ceph-mon[32418]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:17:13.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:12 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:13.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:12 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:13.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:12 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:17:14.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:14 smithi193 ceph-mon[32743]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:17:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:14 smithi072 ceph-mon[29199]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:17:15.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:14 smithi164 ceph-mon[32418]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:17:16.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:16 smithi193 ceph-mon[32743]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:17:17.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:16 smithi072 ceph-mon[29199]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:17:17.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:16 smithi164 ceph-mon[32418]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:17:18.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:18 smithi193 ceph-mon[32743]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:17:19.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:18 smithi072 ceph-mon[29199]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:17:19.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:18 smithi164 ceph-mon[32418]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:17:20.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:20 smithi193 ceph-mon[32743]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:17:21.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:20 smithi072 ceph-mon[29199]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:17:21.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:20 smithi164 ceph-mon[32418]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:17:22.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:22 smithi193 ceph-mon[32743]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:23.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:22 smithi072 ceph-mon[29199]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:23.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:22 smithi164 ceph-mon[32418]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:24.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:24 smithi193 ceph-mon[32743]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:25.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:24 smithi072 ceph-mon[29199]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:25.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:24 smithi164 ceph-mon[32418]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:26.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:26 smithi193 ceph-mon[32743]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:27.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:26 smithi072 ceph-mon[29199]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:27.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:26 smithi164 ceph-mon[32418]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:28.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:28 smithi193 ceph-mon[32743]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:29.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:28 smithi072 ceph-mon[29199]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:29.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:28 smithi164 ceph-mon[32418]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:31.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:30 smithi072 ceph-mon[29199]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:31.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:30 smithi164 ceph-mon[32418]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:31.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:30 smithi193 ceph-mon[32743]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:33.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:32 smithi072 ceph-mon[29199]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:33.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:32 smithi164 ceph-mon[32418]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:33.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:32 smithi193 ceph-mon[32743]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:35.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:34 smithi072 ceph-mon[29199]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:35.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:34 smithi164 ceph-mon[32418]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:35.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:34 smithi193 ceph-mon[32743]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:37.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:36 smithi072 ceph-mon[29199]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:37.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:36 smithi164 ceph-mon[32418]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:37.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:36 smithi193 ceph-mon[32743]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:39.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:38 smithi072 ceph-mon[29199]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:39.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:38 smithi164 ceph-mon[32418]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:39.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:38 smithi193 ceph-mon[32743]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:41.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:40 smithi072 ceph-mon[29199]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:41.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:40 smithi164 ceph-mon[32418]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:41.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:40 smithi193 ceph-mon[32743]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:43.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:42 smithi072 ceph-mon[29199]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:43.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:42 smithi164 ceph-mon[32418]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:43.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:42 smithi193 ceph-mon[32743]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:45.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:44 smithi072 ceph-mon[29199]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:45.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:44 smithi164 ceph-mon[32418]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:45.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:44 smithi193 ceph-mon[32743]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:47.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:46 smithi072 ceph-mon[29199]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:47.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:46 smithi164 ceph-mon[32418]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:47.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:46 smithi193 ceph-mon[32743]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:49.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:48 smithi072 ceph-mon[29199]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:49.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:48 smithi164 ceph-mon[32418]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:49.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:48 smithi193 ceph-mon[32743]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:51.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:50 smithi072 ceph-mon[29199]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:51.033 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:50 smithi193 ceph-mon[32743]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:51.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:50 smithi164 ceph-mon[32418]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:53.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:52 smithi072 ceph-mon[29199]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:53.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:52 smithi164 ceph-mon[32418]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:53.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:52 smithi193 ceph-mon[32743]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:55.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:54 smithi072 ceph-mon[29199]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:55.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:54 smithi164 ceph-mon[32418]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:55.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:54 smithi193 ceph-mon[32743]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:57.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:56 smithi072 ceph-mon[29199]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:57.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:56 smithi164 ceph-mon[32418]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:57.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:56 smithi193 ceph-mon[32743]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:17:59.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:17:58 smithi164 ceph-mon[32418]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:59.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:17:58 smithi193 ceph-mon[32743]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:17:59.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:17:58 smithi072 ceph-mon[29199]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:01.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:00 smithi164 ceph-mon[32418]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:01.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:00 smithi193 ceph-mon[32743]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:01.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:00 smithi072 ceph-mon[29199]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:03.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:02 smithi164 ceph-mon[32418]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:03.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:02 smithi193 ceph-mon[32743]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:03.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:02 smithi072 ceph-mon[29199]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:05.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:04 smithi164 ceph-mon[32418]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:05.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:04 smithi193 ceph-mon[32743]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:05.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:04 smithi072 ceph-mon[29199]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:07.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:06 smithi164 ceph-mon[32418]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:18:07.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:06 smithi193 ceph-mon[32743]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:18:07.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:06 smithi072 ceph-mon[29199]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:18:09.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:08 smithi164 ceph-mon[32418]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:18:09.233 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:08 smithi193 ceph-mon[32743]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:18:09.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:08 smithi072 ceph-mon[29199]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:18:11.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:10 smithi164 ceph-mon[32418]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:18:11.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:10 smithi193 ceph-mon[32743]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:18:11.273 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:10 smithi072 ceph-mon[29199]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:18:13.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:12 smithi164 ceph-mon[32418]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:18:13.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:12 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:13.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:12 smithi193 ceph-mon[32743]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:18:13.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:12 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:13.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:12 smithi072 ceph-mon[29199]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:18:13.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:12 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:14.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:13 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:14.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:13 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:14.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:13 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:18:14.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:13 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:14.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:13 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:14.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:13 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:18:14.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:13 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:14.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:13 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:14.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:13 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:18:15.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:14 smithi193 ceph-mon[32743]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:18:15.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:14 smithi072 ceph-mon[29199]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:18:15.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:14 smithi164 ceph-mon[32418]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:18:17.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:16 smithi193 ceph-mon[32743]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:18:17.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:16 smithi072 ceph-mon[29199]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:18:17.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:16 smithi164 ceph-mon[32418]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:18:19.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:18 smithi193 ceph-mon[32743]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:18:19.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:18 smithi072 ceph-mon[29199]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:18:19.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:18 smithi164 ceph-mon[32418]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:18:21.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:20 smithi193 ceph-mon[32743]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:18:21.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:20 smithi072 ceph-mon[29199]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:18:21.310 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:20 smithi164 ceph-mon[32418]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:18:23.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:22 smithi193 ceph-mon[32743]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:23.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:22 smithi072 ceph-mon[29199]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:23.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:22 smithi164 ceph-mon[32418]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:25.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:24 smithi193 ceph-mon[32743]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:25.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:24 smithi072 ceph-mon[29199]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:25.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:24 smithi164 ceph-mon[32418]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:27.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:26 smithi193 ceph-mon[32743]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:27.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:26 smithi072 ceph-mon[29199]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:27.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:26 smithi164 ceph-mon[32418]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:29.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:28 smithi193 ceph-mon[32743]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:29.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:28 smithi072 ceph-mon[29199]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:29.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:28 smithi164 ceph-mon[32418]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:31.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:30 smithi072 ceph-mon[29199]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:31.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:30 smithi164 ceph-mon[32418]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:31.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:30 smithi193 ceph-mon[32743]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:33.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:32 smithi072 ceph-mon[29199]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:33.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:32 smithi164 ceph-mon[32418]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:33.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:32 smithi193 ceph-mon[32743]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:35.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:34 smithi072 ceph-mon[29199]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:35.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:34 smithi164 ceph-mon[32418]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:35.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:34 smithi193 ceph-mon[32743]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:37.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:36 smithi072 ceph-mon[29199]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:37.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:36 smithi164 ceph-mon[32418]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:37.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:36 smithi193 ceph-mon[32743]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:39.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:38 smithi072 ceph-mon[29199]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:39.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:38 smithi164 ceph-mon[32418]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:39.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:38 smithi193 ceph-mon[32743]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:41.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:40 smithi072 ceph-mon[29199]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:41.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:40 smithi164 ceph-mon[32418]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:41.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:40 smithi193 ceph-mon[32743]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:43.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:43 smithi072 ceph-mon[29199]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:43.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:43 smithi164 ceph-mon[32418]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:43.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:43 smithi193 ceph-mon[32743]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:45.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:45 smithi072 ceph-mon[29199]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:45.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:45 smithi164 ceph-mon[32418]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:45.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:45 smithi193 ceph-mon[32743]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:47.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:47 smithi072 ceph-mon[29199]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:47.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:47 smithi164 ceph-mon[32418]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:47.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:47 smithi193 ceph-mon[32743]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:49.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:49 smithi072 ceph-mon[29199]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:49.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:49 smithi164 ceph-mon[32418]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:49.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:49 smithi193 ceph-mon[32743]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:51.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:51 smithi072 ceph-mon[29199]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:51.284 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:51 smithi193 ceph-mon[32743]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:51.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:51 smithi164 ceph-mon[32418]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:53.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:53 smithi072 ceph-mon[29199]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:53.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:53 smithi164 ceph-mon[32418]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:53.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:53 smithi193 ceph-mon[32743]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:55.274 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:55 smithi072 ceph-mon[29199]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:55.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:55 smithi164 ceph-mon[32418]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:55.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:55 smithi193 ceph-mon[32743]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:57.306 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:57 smithi164 ceph-mon[32418]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:57.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:57 smithi193 ceph-mon[32743]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:57.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:57 smithi072 ceph-mon[29199]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:18:59.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:18:59 smithi164 ceph-mon[32418]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:59.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:18:59 smithi193 ceph-mon[32743]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:18:59.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:18:59 smithi072 ceph-mon[29199]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:01.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:01 smithi164 ceph-mon[32418]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:01.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:01 smithi193 ceph-mon[32743]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:01.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:01 smithi072 ceph-mon[29199]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:03.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:03 smithi193 ceph-mon[32743]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:03.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:03 smithi072 ceph-mon[29199]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:03.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:03 smithi164 ceph-mon[32418]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:05.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:05 smithi193 ceph-mon[32743]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:05.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:05 smithi072 ceph-mon[29199]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:05.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:05 smithi164 ceph-mon[32418]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:07 smithi193 ceph-mon[32743]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:19:07.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:07 smithi072 ceph-mon[29199]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:19:07.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:07 smithi164 ceph-mon[32418]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:19:09.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:09 smithi193 ceph-mon[32743]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:19:09.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:09 smithi072 ceph-mon[29199]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:19:09.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:09 smithi164 ceph-mon[32418]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:19:11.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:11 smithi193 ceph-mon[32743]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:19:11.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:11 smithi072 ceph-mon[29199]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:19:11.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:11 smithi164 ceph-mon[32418]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:19:13.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:13 smithi193 ceph-mon[32743]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:19:13.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:13 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:13.773 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:13 smithi072 ceph-mon[29199]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:19:13.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:13 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:13.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:13 smithi164 ceph-mon[32418]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:19:13.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:13 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:14.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:14 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:14.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:14 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:14.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:14 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:19:14.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:14 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:14.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:14 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:14.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:14 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:19:14.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:14 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:14.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:14 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:14.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:14 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:19:15.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:15 smithi193 ceph-mon[32743]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:19:15.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:15 smithi072 ceph-mon[29199]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:19:15.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:15 smithi164 ceph-mon[32418]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:19:17.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:17 smithi193 ceph-mon[32743]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:19:17.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:17 smithi072 ceph-mon[29199]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:19:17.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:17 smithi164 ceph-mon[32418]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:19:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:19 smithi193 ceph-mon[32743]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:19:19.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:19 smithi072 ceph-mon[29199]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:19:19.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:19 smithi164 ceph-mon[32418]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:19:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:21 smithi193 ceph-mon[32743]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:19:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:21 smithi072 ceph-mon[29199]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:19:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:21 smithi164 ceph-mon[32418]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:19:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:23 smithi193 ceph-mon[32743]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:23.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:23 smithi072 ceph-mon[29199]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:23.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:23 smithi164 ceph-mon[32418]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:25 smithi193 ceph-mon[32743]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:25.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:25 smithi072 ceph-mon[29199]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:25.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:25 smithi164 ceph-mon[32418]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:27 smithi193 ceph-mon[32743]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:27.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:27 smithi072 ceph-mon[29199]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:27.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:27 smithi164 ceph-mon[32418]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:29.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:29 smithi193 ceph-mon[32743]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:29.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:29 smithi072 ceph-mon[29199]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:29.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:29 smithi164 ceph-mon[32418]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:31.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:31 smithi193 ceph-mon[32743]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:31.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:31 smithi072 ceph-mon[29199]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:31.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:31 smithi164 ceph-mon[32418]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:33 smithi193 ceph-mon[32743]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:33.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:33 smithi072 ceph-mon[29199]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:33.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:33 smithi164 ceph-mon[32418]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:35 smithi193 ceph-mon[32743]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:35.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:35 smithi072 ceph-mon[29199]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:35 smithi164 ceph-mon[32418]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:37.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:37 smithi193 ceph-mon[32743]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:37.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:37 smithi072 ceph-mon[29199]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:37.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:37 smithi164 ceph-mon[32418]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:39.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:39 smithi193 ceph-mon[32743]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:39.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:39 smithi072 ceph-mon[29199]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:39.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:39 smithi164 ceph-mon[32418]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:40.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:40 smithi193 ceph-mon[32743]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:40.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:40 smithi072 ceph-mon[29199]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:40.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:40 smithi164 ceph-mon[32418]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:42.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:42 smithi193 ceph-mon[32743]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:43.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:42 smithi072 ceph-mon[29199]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:43.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:42 smithi164 ceph-mon[32418]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:44.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:44 smithi193 ceph-mon[32743]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:45.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:44 smithi072 ceph-mon[29199]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:45.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:44 smithi164 ceph-mon[32418]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:46.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:46 smithi193 ceph-mon[32743]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:47.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:46 smithi072 ceph-mon[29199]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:47.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:46 smithi164 ceph-mon[32418]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:48.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:48 smithi193 ceph-mon[32743]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:49.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:48 smithi072 ceph-mon[29199]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:49.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:48 smithi164 ceph-mon[32418]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:50 smithi193 ceph-mon[32743]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:51.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:50 smithi072 ceph-mon[29199]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:51.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:50 smithi164 ceph-mon[32418]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:52.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:52 smithi193 ceph-mon[32743]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:53.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:52 smithi072 ceph-mon[29199]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:53.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:52 smithi164 ceph-mon[32418]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:54 smithi193 ceph-mon[32743]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:55.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:54 smithi072 ceph-mon[29199]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:55.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:54 smithi164 ceph-mon[32418]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:56.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:56 smithi193 ceph-mon[32743]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:57.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:56 smithi072 ceph-mon[29199]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:57.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:56 smithi164 ceph-mon[32418]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:19:58.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:19:58 smithi193 ceph-mon[32743]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:59.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:19:58 smithi072 ceph-mon[29199]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:19:59.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:19:58 smithi164 ceph-mon[32418]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:00.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:00 smithi193 ceph-mon[32743]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:00.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:00 smithi193 ceph-mon[32743]: overall HEALTH_OK 2024-04-25T13:20:01.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:00 smithi072 ceph-mon[29199]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:01.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:00 smithi072 ceph-mon[29199]: overall HEALTH_OK 2024-04-25T13:20:01.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:00 smithi164 ceph-mon[32418]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:01.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:00 smithi164 ceph-mon[32418]: overall HEALTH_OK 2024-04-25T13:20:02.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:02 smithi193 ceph-mon[32743]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:03.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:02 smithi072 ceph-mon[29199]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:03.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:02 smithi164 ceph-mon[32418]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:04.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:04 smithi193 ceph-mon[32743]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:05.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:04 smithi072 ceph-mon[29199]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:05.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:04 smithi164 ceph-mon[32418]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:06.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:06 smithi193 ceph-mon[32743]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:20:07.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:06 smithi072 ceph-mon[29199]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:20:07.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:06 smithi164 ceph-mon[32418]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:20:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:08 smithi193 ceph-mon[32743]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:20:09.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:08 smithi072 ceph-mon[29199]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:20:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:08 smithi164 ceph-mon[32418]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:20:10.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:10 smithi193 ceph-mon[32743]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:20:10.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:10 smithi193 ceph-mon[32743]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:20:11.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:10 smithi072 ceph-mon[29199]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:20:11.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:10 smithi072 ceph-mon[29199]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:20:11.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:10 smithi164 ceph-mon[32418]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:20:11.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:10 smithi164 ceph-mon[32418]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:20:12.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:12 smithi193 ceph-mon[32743]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:20:13.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:12 smithi072 ceph-mon[29199]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:20:13.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:12 smithi164 ceph-mon[32418]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:20:14.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:14 smithi193 ceph-mon[32743]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:20:14.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:14 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:14.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:14 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:14.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:14 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:14.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:14 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:20:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:14 smithi072 ceph-mon[29199]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:20:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:14 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:14 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:14 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:14 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:20:15.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:14 smithi164 ceph-mon[32418]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:20:15.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:14 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:15.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:14 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:15.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:14 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:15.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:14 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:20:16.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:16 smithi193 ceph-mon[32743]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:20:17.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:16 smithi072 ceph-mon[29199]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:20:17.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:16 smithi164 ceph-mon[32418]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:20:18.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:18 smithi193 ceph-mon[32743]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:20:18.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:18 smithi072 ceph-mon[29199]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:20:18.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:18 smithi164 ceph-mon[32418]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:20:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:19 smithi193 ceph-mon[32743]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:20:19.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:19 smithi072 ceph-mon[29199]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:20:19.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:19 smithi164 ceph-mon[32418]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:20:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:21 smithi193 ceph-mon[32743]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-25T13:20:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:21 smithi072 ceph-mon[29199]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-25T13:20:21.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:21 smithi164 ceph-mon[32418]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-25T13:20:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:23 smithi193 ceph-mon[32743]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:20:23.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:23 smithi072 ceph-mon[29199]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:20:23.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:23 smithi164 ceph-mon[32418]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:20:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:25 smithi193 ceph-mon[32743]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:20:25.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:25 smithi072 ceph-mon[29199]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:20:25.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:25 smithi164 ceph-mon[32418]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:20:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:27 smithi193 ceph-mon[32743]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:20:27.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:27 smithi072 ceph-mon[29199]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:20:27.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:27 smithi164 ceph-mon[32418]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:20:29.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:29 smithi193 ceph-mon[32743]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:20:29.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:29 smithi072 ceph-mon[29199]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:20:29.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:29 smithi164 ceph-mon[32418]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:20:31.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:31 smithi193 ceph-mon[32743]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:31.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:31 smithi072 ceph-mon[29199]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:31.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:31 smithi164 ceph-mon[32418]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:33 smithi193 ceph-mon[32743]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:33.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:33 smithi072 ceph-mon[29199]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:33 smithi164 ceph-mon[32418]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:35 smithi193 ceph-mon[32743]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:35.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:35 smithi072 ceph-mon[29199]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:35.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:35 smithi164 ceph-mon[32418]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:37.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:37 smithi193 ceph-mon[32743]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:37.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:37 smithi072 ceph-mon[29199]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:37.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:37 smithi164 ceph-mon[32418]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:39.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:39 smithi193 ceph-mon[32743]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:39.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:39 smithi072 ceph-mon[29199]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:39.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:39 smithi164 ceph-mon[32418]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:41.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:41 smithi193 ceph-mon[32743]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:41.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:41 smithi072 ceph-mon[29199]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:41.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:41 smithi164 ceph-mon[32418]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:43.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:43 smithi193 ceph-mon[32743]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:43.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:43 smithi072 ceph-mon[29199]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:43.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:43 smithi164 ceph-mon[32418]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:45.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:45 smithi193 ceph-mon[32743]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:45.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:45 smithi072 ceph-mon[29199]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:45.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:45 smithi164 ceph-mon[32418]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:47.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:47 smithi193 ceph-mon[32743]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:47.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:47 smithi072 ceph-mon[29199]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:47.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:47 smithi164 ceph-mon[32418]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:49.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:49 smithi193 ceph-mon[32743]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:49 smithi072 ceph-mon[29199]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:49 smithi164 ceph-mon[32418]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:50 smithi072 ceph-mon[29199]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:50.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:50 smithi164 ceph-mon[32418]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:50 smithi193 ceph-mon[32743]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:52.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:52 smithi193 ceph-mon[32743]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:53.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:52 smithi072 ceph-mon[29199]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:53.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:52 smithi164 ceph-mon[32418]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:54 smithi193 ceph-mon[32743]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:55.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:54 smithi072 ceph-mon[29199]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:55.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:54 smithi164 ceph-mon[32418]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:56.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:56 smithi193 ceph-mon[32743]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:57.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:56 smithi072 ceph-mon[29199]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:57.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:56 smithi164 ceph-mon[32418]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:20:58.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:20:58 smithi193 ceph-mon[32743]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:59.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:20:58 smithi072 ceph-mon[29199]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:20:59.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:20:58 smithi164 ceph-mon[32418]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:00.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:00 smithi193 ceph-mon[32743]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:01.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:00 smithi072 ceph-mon[29199]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:01.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:00 smithi164 ceph-mon[32418]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:02.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:02 smithi193 ceph-mon[32743]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:03.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:02 smithi072 ceph-mon[29199]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:03.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:02 smithi164 ceph-mon[32418]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:04.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:04 smithi193 ceph-mon[32743]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:05.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:04 smithi072 ceph-mon[29199]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:05.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:04 smithi164 ceph-mon[32418]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:06.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:06 smithi193 ceph-mon[32743]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:21:07.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:06 smithi072 ceph-mon[29199]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:21:07.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:06 smithi164 ceph-mon[32418]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:21:08.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:08 smithi193 ceph-mon[32743]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:21:09.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:08 smithi072 ceph-mon[29199]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:21:09.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:08 smithi164 ceph-mon[32418]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:21:10.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:10 smithi193 ceph-mon[32743]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:21:10.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:10 smithi193 ceph-mon[32743]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:21:11.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:10 smithi072 ceph-mon[29199]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:21:11.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:10 smithi072 ceph-mon[29199]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:21:11.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:10 smithi164 ceph-mon[32418]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:21:11.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:10 smithi164 ceph-mon[32418]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:21:12.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:12 smithi193 ceph-mon[32743]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:21:13.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:12 smithi072 ceph-mon[29199]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:21:13.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:12 smithi164 ceph-mon[32418]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:21:14.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:14 smithi193 ceph-mon[32743]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:21:14.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:14 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:14 smithi072 ceph-mon[29199]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:21:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:14 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:15.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:14 smithi164 ceph-mon[32418]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:21:15.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:14 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:15.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:15 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:15.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:15 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:15.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:15 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:21:16.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:15 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:16.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:15 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:16.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:15 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:21:16.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:15 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:16.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:15 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:16.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:15 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:21:17.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:17 smithi164 ceph-mon[32418]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:21:17.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:17 smithi193 ceph-mon[32743]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:21:17.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:17 smithi072 ceph-mon[29199]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:21:18.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:18 smithi193 ceph-mon[32743]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:21:18.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:18 smithi072 ceph-mon[29199]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:21:18.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:18 smithi164 ceph-mon[32418]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:21:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:19 smithi193 ceph-mon[32743]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:21:19.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:19 smithi072 ceph-mon[29199]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:21:19.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:19 smithi164 ceph-mon[32418]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:21:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:21 smithi193 ceph-mon[32743]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-25T13:21:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:21 smithi072 ceph-mon[29199]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-25T13:21:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:21 smithi164 ceph-mon[32418]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-25T13:21:23.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:23 smithi193 ceph-mon[32743]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:21:23.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:23 smithi072 ceph-mon[29199]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:21:23.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:23 smithi164 ceph-mon[32418]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:21:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:25 smithi193 ceph-mon[32743]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:21:25.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:25 smithi072 ceph-mon[29199]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:21:25.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:25 smithi164 ceph-mon[32418]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:21:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:27 smithi193 ceph-mon[32743]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:21:27.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:27 smithi072 ceph-mon[29199]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:21:27.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:27 smithi164 ceph-mon[32418]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:21:29.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:29 smithi193 ceph-mon[32743]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:21:29.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:29 smithi072 ceph-mon[29199]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:21:29.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:29 smithi164 ceph-mon[32418]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:21:31.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:31 smithi193 ceph-mon[32743]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:31.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:31 smithi072 ceph-mon[29199]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:31.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:31 smithi164 ceph-mon[32418]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:33 smithi193 ceph-mon[32743]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:33.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:33 smithi072 ceph-mon[29199]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:33.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:33 smithi164 ceph-mon[32418]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:35 smithi193 ceph-mon[32743]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:35.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:35 smithi072 ceph-mon[29199]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:35 smithi164 ceph-mon[32418]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:37.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:37 smithi193 ceph-mon[32743]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:37.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:37 smithi072 ceph-mon[29199]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:37.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:37 smithi164 ceph-mon[32418]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:39.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:39 smithi193 ceph-mon[32743]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:39.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:39 smithi072 ceph-mon[29199]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:39.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:39 smithi164 ceph-mon[32418]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:41.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:41 smithi193 ceph-mon[32743]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:41.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:41 smithi072 ceph-mon[29199]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:41.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:41 smithi164 ceph-mon[32418]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:43.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:43 smithi193 ceph-mon[32743]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:43.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:43 smithi072 ceph-mon[29199]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:43.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:43 smithi164 ceph-mon[32418]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:45.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:45 smithi193 ceph-mon[32743]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:45.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:45 smithi072 ceph-mon[29199]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:45.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:45 smithi164 ceph-mon[32418]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:47.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:47 smithi193 ceph-mon[32743]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:47.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:47 smithi072 ceph-mon[29199]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:47.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:47 smithi164 ceph-mon[32418]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:49.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:49 smithi193 ceph-mon[32743]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:49 smithi072 ceph-mon[29199]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:49.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:49 smithi164 ceph-mon[32418]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:50.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:50 smithi193 ceph-mon[32743]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:50.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:50 smithi072 ceph-mon[29199]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:50.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:50 smithi164 ceph-mon[32418]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:52.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:52 smithi193 ceph-mon[32743]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:53.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:52 smithi072 ceph-mon[29199]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:53.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:52 smithi164 ceph-mon[32418]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:54.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:54 smithi193 ceph-mon[32743]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:55.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:54 smithi072 ceph-mon[29199]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:55.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:54 smithi164 ceph-mon[32418]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:56.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:56 smithi193 ceph-mon[32743]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:57.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:56 smithi072 ceph-mon[29199]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:57.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:56 smithi164 ceph-mon[32418]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:21:58.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:21:58 smithi193 ceph-mon[32743]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:59.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:21:58 smithi072 ceph-mon[29199]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:21:59.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:21:58 smithi164 ceph-mon[32418]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:00.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:00 smithi193 ceph-mon[32743]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:01.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:00 smithi072 ceph-mon[29199]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:01.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:00 smithi164 ceph-mon[32418]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:02.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:02 smithi193 ceph-mon[32743]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:03.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:02 smithi072 ceph-mon[29199]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:03.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:02 smithi164 ceph-mon[32418]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:04.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:04 smithi193 ceph-mon[32743]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:05.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:04 smithi072 ceph-mon[29199]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:05.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:04 smithi164 ceph-mon[32418]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:06.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:06 smithi193 ceph-mon[32743]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:22:07.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:06 smithi072 ceph-mon[29199]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:22:07.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:06 smithi164 ceph-mon[32418]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:22:08.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:08 smithi193 ceph-mon[32743]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:22:09.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:08 smithi072 ceph-mon[29199]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:22:09.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:08 smithi164 ceph-mon[32418]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:22:10.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:10 smithi193 ceph-mon[32743]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:22:10.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:10 smithi193 ceph-mon[32743]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:22:11.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:10 smithi072 ceph-mon[29199]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:22:11.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:10 smithi072 ceph-mon[29199]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:22:11.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:10 smithi164 ceph-mon[32418]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:22:11.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:10 smithi164 ceph-mon[32418]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:22:12.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:12 smithi193 ceph-mon[32743]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:22:13.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:12 smithi072 ceph-mon[29199]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:22:13.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:12 smithi164 ceph-mon[32418]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:22:14.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:14 smithi193 ceph-mon[32743]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:22:15.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:14 smithi072 ceph-mon[29199]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:22:15.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:14 smithi164 ceph-mon[32418]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:22:15.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:15 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:16.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:15 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:16.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:15 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:16.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:16 smithi193 ceph-mon[32743]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:22:16.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:16 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:16.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:16 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:16.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:16 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:22:17.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:16 smithi072 ceph-mon[29199]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:22:17.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:16 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:17.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:16 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:17.025 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:16 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:22:17.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:16 smithi164 ceph-mon[32418]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:22:17.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:16 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:17.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:16 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:17.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:16 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:22:18.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:18 smithi193 ceph-mon[32743]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:22:18.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:18 smithi072 ceph-mon[29199]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:22:18.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:18 smithi164 ceph-mon[32418]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:22:19.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:19 smithi193 ceph-mon[32743]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:22:19.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:19 smithi072 ceph-mon[29199]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:22:19.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:19 smithi164 ceph-mon[32418]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:22:21.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:21 smithi193 ceph-mon[32743]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-25T13:22:21.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:21 smithi072 ceph-mon[29199]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-25T13:22:21.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:21 smithi164 ceph-mon[32418]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-25T13:22:23.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:23 smithi193 ceph-mon[32743]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:22:23.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:23 smithi072 ceph-mon[29199]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:22:23.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:23 smithi164 ceph-mon[32418]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:22:25.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:25 smithi193 ceph-mon[32743]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:22:25.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:25 smithi072 ceph-mon[29199]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:22:25.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:25 smithi164 ceph-mon[32418]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:22:27.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:27 smithi193 ceph-mon[32743]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:22:27.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:27 smithi072 ceph-mon[29199]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:22:27.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:27 smithi164 ceph-mon[32418]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:22:29.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:29 smithi193 ceph-mon[32743]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:22:29.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:29 smithi072 ceph-mon[29199]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:22:29.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:29 smithi164 ceph-mon[32418]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:22:31.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:31 smithi193 ceph-mon[32743]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:31.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:31 smithi072 ceph-mon[29199]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:31.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:31 smithi164 ceph-mon[32418]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:33.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:33 smithi193 ceph-mon[32743]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:33.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:33 smithi072 ceph-mon[29199]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:33.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:33 smithi164 ceph-mon[32418]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:35.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:35 smithi193 ceph-mon[32743]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:35.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:35 smithi072 ceph-mon[29199]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:35.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:35 smithi164 ceph-mon[32418]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:37.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:37 smithi193 ceph-mon[32743]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:37.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:37 smithi072 ceph-mon[29199]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:37.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:37 smithi164 ceph-mon[32418]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:39.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:39 smithi193 ceph-mon[32743]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:39.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:39 smithi072 ceph-mon[29199]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:39.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:39 smithi164 ceph-mon[32418]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:41.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:41 smithi193 ceph-mon[32743]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:41.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:41 smithi072 ceph-mon[29199]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:41.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:41 smithi164 ceph-mon[32418]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:43.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:43 smithi193 ceph-mon[32743]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:43.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:43 smithi072 ceph-mon[29199]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:43.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:43 smithi164 ceph-mon[32418]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:44 smithi193 ceph-mon[32743]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:44.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:44 smithi072 ceph-mon[29199]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:44.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:44 smithi164 ceph-mon[32418]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:46.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:46 smithi193 ceph-mon[32743]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:47.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:46 smithi072 ceph-mon[29199]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:47.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:46 smithi164 ceph-mon[32418]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:48.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:48 smithi193 ceph-mon[32743]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:49.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:48 smithi072 ceph-mon[29199]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:49.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:48 smithi164 ceph-mon[32418]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:50.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:50 smithi193 ceph-mon[32743]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:51.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:50 smithi072 ceph-mon[29199]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:51.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:50 smithi164 ceph-mon[32418]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:52.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:52 smithi193 ceph-mon[32743]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:53.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:52 smithi072 ceph-mon[29199]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:53.056 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:52 smithi164 ceph-mon[32418]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:54.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:54 smithi193 ceph-mon[32743]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:55.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:54 smithi072 ceph-mon[29199]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:55.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:54 smithi164 ceph-mon[32418]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:56.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:56 smithi193 ceph-mon[32743]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:57.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:56 smithi072 ceph-mon[29199]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:57.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:56 smithi164 ceph-mon[32418]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:22:58.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:22:58 smithi193 ceph-mon[32743]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:59.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:22:58 smithi072 ceph-mon[29199]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:22:59.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:22:58 smithi164 ceph-mon[32418]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:00.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:00 smithi193 ceph-mon[32743]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:01.024 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:00 smithi072 ceph-mon[29199]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:01.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:00 smithi164 ceph-mon[32418]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:03.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:03 smithi193 ceph-mon[32743]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:03.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:03 smithi072 ceph-mon[29199]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:03.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:03 smithi164 ceph-mon[32418]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:05.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:05 smithi193 ceph-mon[32743]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:05.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:05 smithi072 ceph-mon[29199]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:05.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:05 smithi164 ceph-mon[32418]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:06.664 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:23:06.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 -- 172.21.15.72:0/866662817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c1011a0 msgr2=0x7f1d0c101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:06.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 --2- 172.21.15.72:0/866662817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c1011a0 0x7f1d0c101620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009900 tx=0x7f1d0002a400 comp rx=0 tx=0).stop 2024-04-25T13:23:06.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 -- 172.21.15.72:0/866662817 shutdown_connections 2024-04-25T13:23:06.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 --2- 172.21.15.72:0/866662817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d0c101b60 0x7f1d0c10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 --2- 172.21.15.72:0/866662817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c1011a0 0x7f1d0c101620 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 --2- 172.21.15.72:0/866662817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d0c106fb0 0x7f1d0c107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 -- 172.21.15.72:0/866662817 >> 172.21.15.72:0/866662817 conn(0x7f1d0c076c50 msgr2=0x7f1d0c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:06.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 -- 172.21.15.72:0/866662817 shutdown_connections 2024-04-25T13:23:06.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 -- 172.21.15.72:0/866662817 wait complete. 2024-04-25T13:23:06.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 Processor -- start 2024-04-25T13:23:06.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.777+0000 7f1d12431640 1 -- start start 2024-04-25T13:23:06.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d12431640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d0c1011a0 0x7f1d0c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:06.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d12431640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c101b60 0x7f1d0c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:06.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d12431640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d0c106fb0 0x7f1d0c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:06.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d12431640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c112200 con 0x7f1d0c101b60 2024-04-25T13:23:06.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d12431640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c112080 con 0x7f1d0c1011a0 2024-04-25T13:23:06.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d12431640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c112380 con 0x7f1d0c106fb0 2024-04-25T13:23:06.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0b7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c101b60 0x7f1d0c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:06.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0b7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c101b60 0x7f1d0c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37240/0 (socket says 172.21.15.72:37240) 2024-04-25T13:23:06.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0b7fe640 1 -- 172.21.15.72:0/1536318174 learned_addr learned my addr 172.21.15.72:0/1536318174 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:06.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d109ce640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d0c106fb0 0x7f1d0c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:06.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0b7fe640 1 -- 172.21.15.72:0/1536318174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d0c106fb0 msgr2=0x7f1d0c19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:06.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0bfff640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d0c1011a0 0x7f1d0c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:06.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0b7fe640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d0c106fb0 0x7f1d0c19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0b7fe640 1 -- 172.21.15.72:0/1536318174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d0c1011a0 msgr2=0x7f1d0c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:06.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0b7fe640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d0c1011a0 0x7f1d0c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0b7fe640 1 -- 172.21.15.72:0/1536318174 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c19d660 con 0x7f1d0c101b60 2024-04-25T13:23:06.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d0b7fe640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c101b60 0x7f1d0c198c00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1d00006010 tx=0x7f1d000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:06.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.778+0000 7f1d097fa640 1 -- 172.21.15.72:0/1536318174 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0003f030 con 0x7f1d0c101b60 2024-04-25T13:23:06.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.779+0000 7f1d097fa640 1 -- 172.21.15.72:0/1536318174 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d00003800 con 0x7f1d0c101b60 2024-04-25T13:23:06.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.779+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0c19d8f0 con 0x7f1d0c101b60 2024-04-25T13:23:06.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.779+0000 7f1d097fa640 1 -- 172.21.15.72:0/1536318174 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00035070 con 0x7f1d0c101b60 2024-04-25T13:23:06.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.779+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0c1a51e0 con 0x7f1d0c101b60 2024-04-25T13:23:06.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.780+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0c102550 con 0x7f1d0c101b60 2024-04-25T13:23:06.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.781+0000 7f1d097fa640 1 -- 172.21.15.72:0/1536318174 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1d00003cb0 con 0x7f1d0c101b60 2024-04-25T13:23:06.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.781+0000 7f1d097fa640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1cf40a3dc0 0x7f1cf40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:06.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.781+0000 7f1d0bfff640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1cf40a3dc0 0x7f1cf40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:06.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.782+0000 7f1d097fa640 1 -- 172.21.15.72:0/1536318174 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f1d000f14d0 con 0x7f1d0c101b60 2024-04-25T13:23:06.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.782+0000 7f1d0bfff640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1cf40a3dc0 0x7f1cf40a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00b9e0 tx=0x7f1cfc005c40 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:06.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.785+0000 7f1d097fa640 1 -- 172.21.15.72:0/1536318174 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d000bdb50 con 0x7f1d0c101b60 2024-04-25T13:23:06.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.985+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 --> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1d0c062390 con 0x7f1cf40a3dc0 2024-04-25T13:23:06.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.992+0000 7f1d097fa640 1 -- 172.21.15.72:0/1536318174 <== mgr.15276 v2:172.21.15.72:6800/3170992219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f1d0c062390 con 0x7f1cf40a3dc0 2024-04-25T13:23:06.993 INFO:teuthology.orchestra.run.smithi072.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:13:04", "first": "2024-04-25T13:14:00", "last": "2024-04-25T13:23:00", "last_pruned": "2024-04-25T13:23:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T13:23:06.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.994+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1cf40a3dc0 msgr2=0x7f1cf40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:06.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.994+0000 7f1d12431640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1cf40a3dc0 0x7f1cf40a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00b9e0 tx=0x7f1cfc005c40 comp rx=0 tx=0).stop 2024-04-25T13:23:06.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.994+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c101b60 msgr2=0x7f1d0c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:06.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.994+0000 7f1d12431640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c101b60 0x7f1d0c198c00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1d00006010 tx=0x7f1d000042f0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.995+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 shutdown_connections 2024-04-25T13:23:06.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.995+0000 7f1d12431640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f1cf40a3dc0 0x7f1cf40a6280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.995+0000 7f1d12431640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d0c106fb0 0x7f1d0c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.995+0000 7f1d12431640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d0c101b60 0x7f1d0c198c00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.995+0000 7f1d12431640 1 --2- 172.21.15.72:0/1536318174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d0c1011a0 0x7f1d0c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:06.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.995+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 >> 172.21.15.72:0/1536318174 conn(0x7f1d0c076c50 msgr2=0x7f1d0c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:06.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.995+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 shutdown_connections 2024-04-25T13:23:06.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:06.995+0000 7f1d12431640 1 -- 172.21.15.72:0/1536318174 wait complete. 2024-04-25T13:23:07.006 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:23:07.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 -- 172.21.15.72:0/3392250539 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9608101aa0 msgr2=0x7f9608105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 --2- 172.21.15.72:0/3392250539 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9608101aa0 0x7f9608105a90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009900 tx=0x7f95fc02a410 comp rx=0 tx=0).stop 2024-04-25T13:23:07.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 -- 172.21.15.72:0/3392250539 shutdown_connections 2024-04-25T13:23:07.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 --2- 172.21.15.72:0/3392250539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608106170 0x7f960810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 --2- 172.21.15.72:0/3392250539 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9608101aa0 0x7f9608105a90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 --2- 172.21.15.72:0/3392250539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96081010f0 0x7f96081014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 -- 172.21.15.72:0/3392250539 >> 172.21.15.72:0/3392250539 conn(0x7f9608077fd0 msgr2=0x7f96080783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:07.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 -- 172.21.15.72:0/3392250539 shutdown_connections 2024-04-25T13:23:07.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 -- 172.21.15.72:0/3392250539 wait complete. 2024-04-25T13:23:07.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 Processor -- start 2024-04-25T13:23:07.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 -- start start 2024-04-25T13:23:07.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96081010f0 0x7f96081994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608106170 0x7f9608199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9608199fa0 0x7f96081a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9608111090 con 0x7f96081010f0 2024-04-25T13:23:07.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9608110f10 con 0x7f9608106170 2024-04-25T13:23:07.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.106+0000 7f960d859640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9608111210 con 0x7f9608199fa0 2024-04-25T13:23:07.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f9606ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96081010f0 0x7f96081994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f9606ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96081010f0 0x7f96081994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37250/0 (socket says 172.21.15.72:37250) 2024-04-25T13:23:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f9606ffd640 1 -- 172.21.15.72:0/2592952363 learned_addr learned my addr 172.21.15.72:0/2592952363 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f96077fe640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9608199fa0 0x7f96081a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f96067fc640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608106170 0x7f9608199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f9606ffd640 1 -- 172.21.15.72:0/2592952363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9608199fa0 msgr2=0x7f96081a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f9606ffd640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9608199fa0 0x7f96081a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f9606ffd640 1 -- 172.21.15.72:0/2592952363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608106170 msgr2=0x7f9608199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f9606ffd640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608106170 0x7f9608199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f9606ffd640 1 -- 172.21.15.72:0/2592952363 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96081a4d20 con 0x7f96081010f0 2024-04-25T13:23:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f9606ffd640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96081010f0 0x7f96081994d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c920 tx=0x7f95f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f960c857640 1 -- 172.21.15.72:0/2592952363 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8007c70 con 0x7f96081010f0 2024-04-25T13:23:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f960c857640 1 -- 172.21.15.72:0/2592952363 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f8007e10 con 0x7f96081010f0 2024-04-25T13:23:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96081a4fb0 con 0x7f96081010f0 2024-04-25T13:23:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f960c857640 1 -- 172.21.15.72:0/2592952363 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8018820 con 0x7f96081010f0 2024-04-25T13:23:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.107+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96081a5480 con 0x7f96081010f0 2024-04-25T13:23:07.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.108+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960804fce0 con 0x7f96081010f0 2024-04-25T13:23:07.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.109+0000 7f960c857640 1 -- 172.21.15.72:0/2592952363 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f95f800f450 con 0x7f96081010f0 2024-04-25T13:23:07.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.110+0000 7f960c857640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f95f00a3dc0 0x7f95f00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.110+0000 7f960c857640 1 -- 172.21.15.72:0/2592952363 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f95f80d0a30 con 0x7f96081010f0 2024-04-25T13:23:07.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.110+0000 7f96067fc640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f95f00a3dc0 0x7f95f00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.110+0000 7f96067fc640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f95f00a3dc0 0x7f95f00a6280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc031400 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:07.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.114+0000 7f960c857640 1 -- 172.21.15.72:0/2592952363 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95f8014030 con 0x7f96081010f0 2024-04-25T13:23:07.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.314+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 --> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9608101aa0 con 0x7f95f00a3dc0 2024-04-25T13:23:07.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.321+0000 7f960c857640 1 -- 172.21.15.72:0/2592952363 <== mgr.15276 v2:172.21.15.72:6800/3170992219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9608101aa0 con 0x7f95f00a3dc0 2024-04-25T13:23:07.322 INFO:teuthology.orchestra.run.smithi072.stdout:/ 1m 6m3h 2024-04-25T13:23:07.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f95f00a3dc0 msgr2=0x7f95f00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f95f00a3dc0 0x7f95f00a6280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc031400 comp rx=0 tx=0).stop 2024-04-25T13:23:07.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96081010f0 msgr2=0x7f96081994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96081010f0 0x7f96081994d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c920 tx=0x7f95f800cde0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 shutdown_connections 2024-04-25T13:23:07.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f95f00a3dc0 0x7f95f00a6280 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9608199fa0 0x7f96081a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608106170 0x7f9608199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 --2- 172.21.15.72:0/2592952363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96081010f0 0x7f96081994d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 >> 172.21.15.72:0/2592952363 conn(0x7f9608077fd0 msgr2=0x7f96080fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:07.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 shutdown_connections 2024-04-25T13:23:07.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.323+0000 7f960d859640 1 -- 172.21.15.72:0/2592952363 wait complete. 2024-04-25T13:23:07.335 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T13:23:07.347 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:23:07.449 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 -- 172.21.15.72:0/2117495413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5801014c0 msgr2=0x7fa580106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.449 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 --2- 172.21.15.72:0/2117495413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5801014c0 0x7fa580106bf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa568009900 tx=0x7fa56802a410 comp rx=0 tx=0).stop 2024-04-25T13:23:07.449 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 -- 172.21.15.72:0/2117495413 shutdown_connections 2024-04-25T13:23:07.449 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 --2- 172.21.15.72:0/2117495413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5801072d0 0x7fa58010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.449 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 --2- 172.21.15.72:0/2117495413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5801014c0 0x7fa580106bf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.449 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 --2- 172.21.15.72:0/2117495413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa580100b10 0x7fa580100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.449 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 -- 172.21.15.72:0/2117495413 >> 172.21.15.72:0/2117495413 conn(0x7fa580076130 msgr2=0x7fa580076540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:07.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 -- 172.21.15.72:0/2117495413 shutdown_connections 2024-04-25T13:23:07.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 -- 172.21.15.72:0/2117495413 wait complete. 2024-04-25T13:23:07.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 Processor -- start 2024-04-25T13:23:07.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.446+0000 7fa585aae640 1 -- start start 2024-04-25T13:23:07.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa585aae640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa580100b10 0x7fa580198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa585aae640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5801072d0 0x7fa580198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa585aae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa58019cfe0 0x7fa58019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa585aae640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa580112240 con 0x7fa580100b10 2024-04-25T13:23:07.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa585aae640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801120c0 con 0x7fa5801072d0 2024-04-25T13:23:07.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa585aae640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801123c0 con 0x7fa58019cfe0 2024-04-25T13:23:07.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa580100b10 0x7fa580198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa580100b10 0x7fa580198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37270/0 (socket says 172.21.15.72:37270) 2024-04-25T13:23:07.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57f7fe640 1 -- 172.21.15.72:0/446452062 learned_addr learned my addr 172.21.15.72:0/446452062 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:07.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa577fff640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5801072d0 0x7fa580198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57ffff640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa58019cfe0 0x7fa58019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57f7fe640 1 -- 172.21.15.72:0/446452062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa58019cfe0 msgr2=0x7fa58019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57f7fe640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa58019cfe0 0x7fa58019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57f7fe640 1 -- 172.21.15.72:0/446452062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5801072d0 msgr2=0x7fa580198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57f7fe640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5801072d0 0x7fa580198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57f7fe640 1 -- 172.21.15.72:0/446452062 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58019dad0 con 0x7fa580100b10 2024-04-25T13:23:07.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57f7fe640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa580100b10 0x7fa580198720 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa57000c920 tx=0x7fa57000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:07.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57d7fa640 1 -- 172.21.15.72:0/446452062 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570007c70 con 0x7fa580100b10 2024-04-25T13:23:07.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57d7fa640 1 -- 172.21.15.72:0/446452062 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa570007e10 con 0x7fa580100b10 2024-04-25T13:23:07.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58019ddc0 con 0x7fa580100b10 2024-04-25T13:23:07.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa57d7fa640 1 -- 172.21.15.72:0/446452062 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570018820 con 0x7fa580100b10 2024-04-25T13:23:07.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.447+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5801a5580 con 0x7fa580100b10 2024-04-25T13:23:07.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.448+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5800691c0 con 0x7fa580100b10 2024-04-25T13:23:07.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.449+0000 7fa57d7fa640 1 -- 172.21.15.72:0/446452062 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa57000f450 con 0x7fa580100b10 2024-04-25T13:23:07.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.450+0000 7fa57d7fa640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa56c0a3d20 0x7fa56c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.450+0000 7fa577fff640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa56c0a3d20 0x7fa56c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.450+0000 7fa57d7fa640 1 -- 172.21.15.72:0/446452062 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fa5700d0cf0 con 0x7fa580100b10 2024-04-25T13:23:07.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.451+0000 7fa577fff640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa56c0a3d20 0x7fa56c0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa568000c00 tx=0x7fa568032ca0 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:07.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.454+0000 7fa57d7fa640 1 -- 172.21.15.72:0/446452062 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa570014030 con 0x7fa580100b10 2024-04-25T13:23:07.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.632+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa58010a4e0 con 0x7fa580100b10 2024-04-25T13:23:07.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.633+0000 7fa57d7fa640 1 -- 172.21.15.72:0/446452062 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23208 (secure 0 0 0) 0x7fa57009d3a0 con 0x7fa580100b10 2024-04-25T13:23:07.635 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:23:07.635 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":97,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:22:17.402830+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/3348457169":"2024-04-26T13:13:03.464543+0000","172.21.15.72:0/4004940606":"2024-04-26T13:13:03.464543+0000","172.21.15.72:0/3501290813":"2024-04-26T13:13:03.464543+0000","172.21.15.72:6800/3348457169":"2024-04-26T13:13:03.464543+0000","172.21.15.72:0/1633858068":"2024-04-26T13:13:03.464543+0000","172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2743588764":"2024-04-26T13:13:03.464543+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":4,"length":1},{"begin":13,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:23:07.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa56c0a3d20 msgr2=0x7fa56c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa56c0a3d20 0x7fa56c0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa568000c00 tx=0x7fa568032ca0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa580100b10 msgr2=0x7fa580198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa580100b10 0x7fa580198720 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa57000c920 tx=0x7fa57000cde0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 shutdown_connections 2024-04-25T13:23:07.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa56c0a3d20 0x7fa56c0a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa58019cfe0 0x7fa58019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5801072d0 0x7fa580198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 --2- 172.21.15.72:0/446452062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa580100b10 0x7fa580198720 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.635+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 >> 172.21.15.72:0/446452062 conn(0x7fa580076130 msgr2=0x7fa5800feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:07.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.636+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 shutdown_connections 2024-04-25T13:23:07.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.636+0000 7fa585aae640 1 -- 172.21.15.72:0/446452062 wait complete. 2024-04-25T13:23:07.646 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:07 smithi072 ceph-mon[29199]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:23:07.647 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:06:33.150709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:11:02.352089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:11:03.565033+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:23:07.648 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:23:07.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:07 smithi193 ceph-mon[32743]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:23:07.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.748+0000 7f2cf78d0640 1 -- 172.21.15.72:0/1916209282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0102b40 msgr2=0x7f2cf0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.748+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/1916209282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0102b40 0x7f2cf0106b80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009900 tx=0x7f2ce002a410 comp rx=0 tx=0).stop 2024-04-25T13:23:07.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.748+0000 7f2cf78d0640 1 -- 172.21.15.72:0/1916209282 shutdown_connections 2024-04-25T13:23:07.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.748+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/1916209282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cf0107260 0x7f2cf010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.748+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/1916209282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0102b40 0x7f2cf0106b80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.748+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/1916209282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cf0102190 0x7f2cf0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.748+0000 7f2cf78d0640 1 -- 172.21.15.72:0/1916209282 >> 172.21.15.72:0/1916209282 conn(0x7f2cf00793a0 msgr2=0x7f2cf00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:07.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.748+0000 7f2cf78d0640 1 -- 172.21.15.72:0/1916209282 shutdown_connections 2024-04-25T13:23:07.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.748+0000 7f2cf78d0640 1 -- 172.21.15.72:0/1916209282 wait complete. 2024-04-25T13:23:07.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf78d0640 1 Processor -- start 2024-04-25T13:23:07.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf78d0640 1 -- start start 2024-04-25T13:23:07.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf78d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cf0102190 0x7f2cf019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf78d0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0107260 0x7f2cf019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf78d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cf0198650 0x7f2cf0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf78d0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf01121a0 con 0x7f2cf0107260 2024-04-25T13:23:07.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf78d0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0112020 con 0x7f2cf0102190 2024-04-25T13:23:07.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf78d0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0112320 con 0x7f2cf0198650 2024-04-25T13:23:07.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf4e6b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0107260 0x7f2cf019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf4e6b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0107260 0x7f2cf019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37296/0 (socket says 172.21.15.72:37296) 2024-04-25T13:23:07.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf4e6b640 1 -- 172.21.15.72:0/3026152026 learned_addr learned my addr 172.21.15.72:0/3026152026 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf566c640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cf0102190 0x7f2cf019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf4e6b640 1 -- 172.21.15.72:0/3026152026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cf0198650 msgr2=0x7f2cf0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf4e6b640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cf0198650 0x7f2cf0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf4e6b640 1 -- 172.21.15.72:0/3026152026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cf0102190 msgr2=0x7f2cf019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf4e6b640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cf0102190 0x7f2cf019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.749+0000 7f2cf4e6b640 1 -- 172.21.15.72:0/3026152026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf0199170 con 0x7f2cf0107260 2024-04-25T13:23:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.750+0000 7f2cf4e6b640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0107260 0x7f2cf019e990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0032fd0 tx=0x7f2ce0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.750+0000 7f2ce67fc640 1 -- 172.21.15.72:0/3026152026 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce003f030 con 0x7f2cf0107260 2024-04-25T13:23:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.750+0000 7f2ce67fc640 1 -- 172.21.15.72:0/3026152026 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ce0004490 con 0x7f2cf0107260 2024-04-25T13:23:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.750+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf0199400 con 0x7f2cf0107260 2024-04-25T13:23:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.750+0000 7f2ce67fc640 1 -- 172.21.15.72:0/3026152026 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0035070 con 0x7f2cf0107260 2024-04-25T13:23:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.750+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a55a0 con 0x7f2cf0107260 2024-04-25T13:23:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.751+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf00691c0 con 0x7f2cf0107260 2024-04-25T13:23:07.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.752+0000 7f2ce67fc640 1 -- 172.21.15.72:0/3026152026 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ce0042030 con 0x7f2cf0107260 2024-04-25T13:23:07.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.752+0000 7f2ce67fc640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2cdc0a3d20 0x7f2cdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:07.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.753+0000 7f2cf566c640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2cdc0a3d20 0x7f2cdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.753+0000 7f2ce67fc640 1 -- 172.21.15.72:0/3026152026 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f2ce00f0d20 con 0x7f2cf0107260 2024-04-25T13:23:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.753+0000 7f2cf566c640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2cdc0a3d20 0x7f2cdc0a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005fd0 tx=0x7f2cec009680 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.757+0000 7f2ce67fc640 1 -- 172.21.15.72:0/3026152026 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ce00bd3a0 con 0x7f2cf0107260 2024-04-25T13:23:07.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:07 smithi164 ceph-mon[32418]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:23:07.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.931+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cf0102b40 con 0x7f2cf0107260 2024-04-25T13:23:07.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.932+0000 7f2ce67fc640 1 -- 172.21.15.72:0/3026152026 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f2ce00bcca0 con 0x7f2cf0107260 2024-04-25T13:23:07.933 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-25T13:23:07.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2cdc0a3d20 msgr2=0x7f2cdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2cdc0a3d20 0x7f2cdc0a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005fd0 tx=0x7f2cec009680 comp rx=0 tx=0).stop 2024-04-25T13:23:07.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0107260 msgr2=0x7f2cf019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:07.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0107260 0x7f2cf019e990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0032fd0 tx=0x7f2ce0004270 comp rx=0 tx=0).stop 2024-04-25T13:23:07.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 shutdown_connections 2024-04-25T13:23:07.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2cdc0a3d20 0x7f2cdc0a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cf0198650 0x7f2cf0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cf0107260 0x7f2cf019e990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 --2- 172.21.15.72:0/3026152026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cf0102190 0x7f2cf019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:07.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 >> 172.21.15.72:0/3026152026 conn(0x7f2cf00793a0 msgr2=0x7f2cf01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:07.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 shutdown_connections 2024-04-25T13:23:07.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:07.934+0000 7f2cf78d0640 1 -- 172.21.15.72:0/3026152026 wait complete. 2024-04-25T13:23:07.945 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:23:08.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.046+0000 7f9e67d76640 1 -- 172.21.15.72:0/4242192734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101a20 msgr2=0x7f9e60105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.046+0000 7f9e67d76640 1 --2- 172.21.15.72:0/4242192734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101a20 0x7f9e60105a60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009900 tx=0x7f9e5402a410 comp rx=0 tx=0).stop 2024-04-25T13:23:08.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.046+0000 7f9e67d76640 1 -- 172.21.15.72:0/4242192734 shutdown_connections 2024-04-25T13:23:08.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.046+0000 7f9e67d76640 1 --2- 172.21.15.72:0/4242192734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e60106140 0x7f9e6010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.046+0000 7f9e67d76640 1 --2- 172.21.15.72:0/4242192734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101a20 0x7f9e60105a60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.046+0000 7f9e67d76640 1 --2- 172.21.15.72:0/4242192734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e60101070 0x7f9e60101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.046+0000 7f9e67d76640 1 -- 172.21.15.72:0/4242192734 >> 172.21.15.72:0/4242192734 conn(0x7f9e600771d0 msgr2=0x7f9e600775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:08.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 -- 172.21.15.72:0/4242192734 shutdown_connections 2024-04-25T13:23:08.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 -- 172.21.15.72:0/4242192734 wait complete. 2024-04-25T13:23:08.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 Processor -- start 2024-04-25T13:23:08.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 -- start start 2024-04-25T13:23:08.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101070 0x7f9e60198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e60106140 0x7f9e60198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e6019cfe0 0x7f9e6019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60112180 con 0x7f9e60101070 2024-04-25T13:23:08.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60112000 con 0x7f9e6019cfe0 2024-04-25T13:23:08.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e67d76640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60112300 con 0x7f9e60106140 2024-04-25T13:23:08.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e65b12640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101070 0x7f9e60198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e65b12640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101070 0x7f9e60198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37306/0 (socket says 172.21.15.72:37306) 2024-04-25T13:23:08.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.047+0000 7f9e65b12640 1 -- 172.21.15.72:0/3216530499 learned_addr learned my addr 172.21.15.72:0/3216530499 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:08.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e65311640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e60106140 0x7f9e60198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e66313640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e6019cfe0 0x7f9e6019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e65b12640 1 -- 172.21.15.72:0/3216530499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e60106140 msgr2=0x7f9e60198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e65b12640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e60106140 0x7f9e60198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e65b12640 1 -- 172.21.15.72:0/3216530499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e6019cfe0 msgr2=0x7f9e6019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e65b12640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e6019cfe0 0x7f9e6019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e65b12640 1 -- 172.21.15.72:0/3216530499 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6019dad0 con 0x7f9e60101070 2024-04-25T13:23:08.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e65311640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e60106140 0x7f9e60198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:23:08.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e65b12640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101070 0x7f9e60198720 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000eab0 tx=0x7f9e5000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:08.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e5effd640 1 -- 172.21.15.72:0/3216530499 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000ce40 con 0x7f9e60101070 2024-04-25T13:23:08.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e5effd640 1 -- 172.21.15.72:0/3216530499 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e50011920 con 0x7f9e60101070 2024-04-25T13:23:08.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6019ddc0 con 0x7f9e60101070 2024-04-25T13:23:08.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e5effd640 1 -- 172.21.15.72:0/3216530499 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50010a30 con 0x7f9e60101070 2024-04-25T13:23:08.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.048+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e601a5580 con 0x7f9e60101070 2024-04-25T13:23:08.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.049+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e600691c0 con 0x7f9e60101070 2024-04-25T13:23:08.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.050+0000 7f9e5effd640 1 -- 172.21.15.72:0/3216530499 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9e50010bd0 con 0x7f9e60101070 2024-04-25T13:23:08.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.050+0000 7f9e5effd640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e480a3d20 0x7f9e480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.051+0000 7f9e5effd640 1 -- 172.21.15.72:0/3216530499 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f9e500d0f30 con 0x7f9e60101070 2024-04-25T13:23:08.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.051+0000 7f9e65311640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e480a3d20 0x7f9e480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.051+0000 7f9e65311640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e480a3d20 0x7f9e480a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9e54000c00 tx=0x7f9e54009610 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:08.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.055+0000 7f9e5effd640 1 -- 172.21.15.72:0/3216530499 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e50016030 con 0x7f9e60101070 2024-04-25T13:23:08.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.231+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9e601093d0 con 0x7f9e60101070 2024-04-25T13:23:08.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.231+0000 7f9e5effd640 1 -- 172.21.15.72:0/3216530499 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f9e5009d5b0 con 0x7f9e60101070 2024-04-25T13:23:08.232 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-25T13:23:08.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e480a3d20 msgr2=0x7f9e480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e480a3d20 0x7f9e480a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9e54000c00 tx=0x7f9e54009610 comp rx=0 tx=0).stop 2024-04-25T13:23:08.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101070 msgr2=0x7f9e60198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101070 0x7f9e60198720 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000eab0 tx=0x7f9e5000ef70 comp rx=0 tx=0).stop 2024-04-25T13:23:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 shutdown_connections 2024-04-25T13:23:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e480a3d20 0x7f9e480a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e6019cfe0 0x7f9e6019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e60106140 0x7f9e60198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 --2- 172.21.15.72:0/3216530499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e60101070 0x7f9e60198720 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 >> 172.21.15.72:0/3216530499 conn(0x7f9e600771d0 msgr2=0x7f9e600feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 shutdown_connections 2024-04-25T13:23:08.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.233+0000 7f9e67d76640 1 -- 172.21.15.72:0/3216530499 wait complete. 2024-04-25T13:23:08.245 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:23:08.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.345+0000 7fcf8ff30640 1 -- 172.21.15.72:0/2852315341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88102b40 msgr2=0x7fcf88106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.345+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/2852315341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88102b40 0x7fcf88106b80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009900 tx=0x7fcf7802a410 comp rx=0 tx=0).stop 2024-04-25T13:23:08.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.345+0000 7fcf8ff30640 1 -- 172.21.15.72:0/2852315341 shutdown_connections 2024-04-25T13:23:08.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.345+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/2852315341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf88107260 0x7fcf8810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.345+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/2852315341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88102b40 0x7fcf88106b80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.345+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/2852315341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf88102190 0x7fcf88102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.345+0000 7fcf8ff30640 1 -- 172.21.15.72:0/2852315341 >> 172.21.15.72:0/2852315341 conn(0x7fcf880793a0 msgr2=0x7fcf880797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:08.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.345+0000 7fcf8ff30640 1 -- 172.21.15.72:0/2852315341 shutdown_connections 2024-04-25T13:23:08.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.345+0000 7fcf8ff30640 1 -- 172.21.15.72:0/2852315341 wait complete. 2024-04-25T13:23:08.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8ff30640 1 Processor -- start 2024-04-25T13:23:08.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8ff30640 1 -- start start 2024-04-25T13:23:08.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8ff30640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf88102190 0x7fcf881a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8ff30640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88107260 0x7fcf881a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8ff30640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf8819a7a0 0x7fcf8819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8ff30640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcf88112180 con 0x7fcf88107260 2024-04-25T13:23:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8ff30640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf88112000 con 0x7fcf8819a7a0 2024-04-25T13:23:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8ff30640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf88112300 con 0x7fcf88102190 2024-04-25T13:23:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8d4cb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88107260 0x7fcf881a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8d4cb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88107260 0x7fcf881a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37318/0 (socket says 172.21.15.72:37318) 2024-04-25T13:23:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8d4cb640 1 -- 172.21.15.72:0/1989606079 learned_addr learned my addr 172.21.15.72:0/1989606079 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8dccc640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf88102190 0x7fcf881a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8e4cd640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf8819a7a0 0x7fcf8819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8d4cb640 1 -- 172.21.15.72:0/1989606079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf88102190 msgr2=0x7fcf881a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8d4cb640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf88102190 0x7fcf881a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8d4cb640 1 -- 172.21.15.72:0/1989606079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf8819a7a0 msgr2=0x7fcf8819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8d4cb640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf8819a7a0 0x7fcf8819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8d4cb640 1 -- 172.21.15.72:0/1989606079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8819b330 con 0x7fcf88107260 2024-04-25T13:23:08.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.346+0000 7fcf8d4cb640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88107260 0x7fcf881a0b70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcf78032fd0 tx=0x7fcf78004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:08.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.347+0000 7fcf7effd640 1 -- 172.21.15.72:0/1989606079 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7803f030 con 0x7fcf88107260 2024-04-25T13:23:08.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.347+0000 7fcf7effd640 1 -- 172.21.15.72:0/1989606079 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf78004490 con 0x7fcf88107260 2024-04-25T13:23:08.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.347+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8819b5c0 con 0x7fcf88107260 2024-04-25T13:23:08.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.347+0000 7fcf7effd640 1 -- 172.21.15.72:0/1989606079 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78035070 con 0x7fcf88107260 2024-04-25T13:23:08.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.347+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf881a7770 con 0x7fcf88107260 2024-04-25T13:23:08.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.348+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf88103660 con 0x7fcf88107260 2024-04-25T13:23:08.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.349+0000 7fcf7effd640 1 -- 172.21.15.72:0/1989606079 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf78042030 con 0x7fcf88107260 2024-04-25T13:23:08.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.349+0000 7fcf7effd640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fcf740a3d20 0x7fcf740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.349+0000 7fcf8dccc640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fcf740a3d20 0x7fcf740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.349+0000 7fcf7effd640 1 -- 172.21.15.72:0/1989606079 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7fcf780f1070 con 0x7fcf88107260 2024-04-25T13:23:08.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.350+0000 7fcf8dccc640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fcf740a3d20 0x7fcf740a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf840058c0 tx=0x7fcf8400a2d0 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:08.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.353+0000 7fcf7effd640 1 -- 172.21.15.72:0/1989606079 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf780bd0a0 con 0x7fcf88107260 2024-04-25T13:23:08.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:08 smithi072 ceph-mon[29199]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:23:08.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:08 smithi072 ceph-mon[29199]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:23:08.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:08 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/446452062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:08 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3026152026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:08.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:08 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3216530499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:23:08.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.535+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcf88102b40 con 0x7fcf88107260 2024-04-25T13:23:08.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.536+0000 7fcf7effd640 1 -- 172.21.15.72:0/1989606079 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fcf780dd0a0 con 0x7fcf88107260 2024-04-25T13:23:08.537 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-25T13:23:08.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fcf740a3d20 msgr2=0x7fcf740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fcf740a3d20 0x7fcf740a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf840058c0 tx=0x7fcf8400a2d0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88107260 msgr2=0x7fcf881a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88107260 0x7fcf881a0b70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcf78032fd0 tx=0x7fcf78004270 comp rx=0 tx=0).stop 2024-04-25T13:23:08.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 shutdown_connections 2024-04-25T13:23:08.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fcf740a3d20 0x7fcf740a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf8819a7a0 0x7fcf8819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf88107260 0x7fcf881a0b70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 --2- 172.21.15.72:0/1989606079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf88102190 0x7fcf881a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 >> 172.21.15.72:0/1989606079 conn(0x7fcf880793a0 msgr2=0x7fcf88100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:08.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 shutdown_connections 2024-04-25T13:23:08.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.538+0000 7fcf8ff30640 1 -- 172.21.15.72:0/1989606079 wait complete. 2024-04-25T13:23:08.549 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T13:23:08.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 -- 172.21.15.72:0/4217350655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4102b50 msgr2=0x7f65c4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 --2- 172.21.15.72:0/4217350655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4102b50 0x7f65c4106b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009900 tx=0x7f65b002a410 comp rx=0 tx=0).stop 2024-04-25T13:23:08.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 -- 172.21.15.72:0/4217350655 shutdown_connections 2024-04-25T13:23:08.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 --2- 172.21.15.72:0/4217350655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65c4107270 0x7f65c410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 --2- 172.21.15.72:0/4217350655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4102b50 0x7f65c4106b90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 --2- 172.21.15.72:0/4217350655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c41021a0 0x7f65c4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 -- 172.21.15.72:0/4217350655 >> 172.21.15.72:0/4217350655 conn(0x7f65c40fd9e0 msgr2=0x7f65c40ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:08.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 -- 172.21.15.72:0/4217350655 shutdown_connections 2024-04-25T13:23:08.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 -- 172.21.15.72:0/4217350655 wait complete. 2024-04-25T13:23:08.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 Processor -- start 2024-04-25T13:23:08.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.648+0000 7f65cab19640 1 -- start start 2024-04-25T13:23:08.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65cab19640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c41021a0 0x7f65c419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65cab19640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4107270 0x7f65c419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65cab19640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65c41985f0 0x7f65c4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65cab19640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4112210 con 0x7f65c4107270 2024-04-25T13:23:08.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65cab19640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4112090 con 0x7f65c41985f0 2024-04-25T13:23:08.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65cab19640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4112390 con 0x7f65c41021a0 2024-04-25T13:23:08.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4107270 0x7f65c419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4107270 0x7f65c419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37334/0 (socket says 172.21.15.72:37334) 2024-04-25T13:23:08.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c3fff640 1 -- 172.21.15.72:0/314884638 learned_addr learned my addr 172.21.15.72:0/314884638 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:08.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c88b5640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c41021a0 0x7f65c419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c3fff640 1 -- 172.21.15.72:0/314884638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c41021a0 msgr2=0x7f65c419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c3fff640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c41021a0 0x7f65c419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c3fff640 1 -- 172.21.15.72:0/314884638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65c41985f0 msgr2=0x7f65c4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:08.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c3fff640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65c41985f0 0x7f65c4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:08.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c3fff640 1 -- 172.21.15.72:0/314884638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c4199110 con 0x7f65c4107270 2024-04-25T13:23:08.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c88b5640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c41021a0 0x7f65c419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:23:08.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c3fff640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4107270 0x7f65c419e980 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f65b0031a90 tx=0x7f65b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:08.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c1ffb640 1 -- 172.21.15.72:0/314884638 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b003f030 con 0x7f65c4107270 2024-04-25T13:23:08.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c1ffb640 1 -- 172.21.15.72:0/314884638 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b0003cb0 con 0x7f65c4107270 2024-04-25T13:23:08.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c41993a0 con 0x7f65c4107270 2024-04-25T13:23:08.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.649+0000 7f65c1ffb640 1 -- 172.21.15.72:0/314884638 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0035070 con 0x7f65c4107270 2024-04-25T13:23:08.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.650+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41a5590 con 0x7f65c4107270 2024-04-25T13:23:08.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.651+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c4068d70 con 0x7f65c4107270 2024-04-25T13:23:08.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.652+0000 7f65c1ffb640 1 -- 172.21.15.72:0/314884638 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65b0042030 con 0x7f65c4107270 2024-04-25T13:23:08.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.652+0000 7f65c1ffb640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f65b40a3d20 0x7f65b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:08.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.652+0000 7f65c88b5640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f65b40a3d20 0x7f65b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:08.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.652+0000 7f65c1ffb640 1 -- 172.21.15.72:0/314884638 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8550+0+0 (secure 0 0 0) 0x7f65b00f0e80 con 0x7f65c4107270 2024-04-25T13:23:08.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.653+0000 7f65c88b5640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f65b40a3d20 0x7f65b40a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006fd0 tx=0x7f65b8008040 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:08.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.657+0000 7f65c1ffb640 1 -- 172.21.15.72:0/314884638 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65b00bd500 con 0x7f65c4107270 2024-04-25T13:23:08.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:08 smithi193 ceph-mon[32743]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:23:08.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:08 smithi193 ceph-mon[32743]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:23:08.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:08 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/446452062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:08.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:08 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3026152026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:08.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:08 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3216530499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:23:08.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:08 smithi164 ceph-mon[32418]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:23:08.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:08 smithi164 ceph-mon[32418]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:23:08.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:08 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/446452062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:08.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:08 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3026152026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:08.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:08 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3216530499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:23:08.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.880+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f65c4102b50 con 0x7f65c4107270 2024-04-25T13:23:08.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.888+0000 7f65c1ffb640 1 -- 172.21.15.72:0/314884638 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746061 (secure 0 0 0) 0x7f65b00bce00 con 0x7f65c4107270 2024-04-25T13:23:08.900 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:23:08.900 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:23:08.900 INFO:teuthology.orchestra.run.smithi072.stdout: "epoch": 26, 2024-04-25T13:23:08.901 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.901 INFO:teuthology.orchestra.run.smithi072.stdout: "active_gid": 15276, 2024-04-25T13:23:08.901 INFO:teuthology.orchestra.run.smithi072.stdout: "active_name": "x", 2024-04-25T13:23:08.901 INFO:teuthology.orchestra.run.smithi072.stdout: "active_addrs": { 2024-04-25T13:23:08.901 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-25T13:23:08.901 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:08.901 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-25T13:23:08.901 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:6800", 2024-04-25T13:23:08.902 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3170992219 2024-04-25T13:23:08.902 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.902 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:08.902 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v1", 2024-04-25T13:23:08.902 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:6801", 2024-04-25T13:23:08.902 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3170992219 2024-04-25T13:23:08.902 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:08.903 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:08.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.903 INFO:teuthology.orchestra.run.smithi072.stdout: "active_addr": "172.21.15.72:6801/3170992219", 2024-04-25T13:23:08.903 INFO:teuthology.orchestra.run.smithi072.stdout: "active_change": "2024-04-25T13:13:03.464801+0000", 2024-04-25T13:23:08.903 INFO:teuthology.orchestra.run.smithi072.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T13:23:08.903 INFO:teuthology.orchestra.run.smithi072.stdout: "available": true, 2024-04-25T13:23:08.903 INFO:teuthology.orchestra.run.smithi072.stdout: "standbys": [ 2024-04-25T13:23:08.904 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:08.904 INFO:teuthology.orchestra.run.smithi072.stdout: "gid": 24848, 2024-04-25T13:23:08.904 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "y", 2024-04-25T13:23:08.904 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:23:08.904 INFO:teuthology.orchestra.run.smithi072.stdout: "available_modules": [ 2024-04-25T13:23:08.904 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:08.904 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "alerts", 2024-04-25T13:23:08.904 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:08.905 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:08.905 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:08.905 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:08.905 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:08.905 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:08.905 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.905 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.906 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:08.906 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.906 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.906 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.906 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:23:08.906 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.906 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.907 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.907 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:08.907 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:08.907 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.907 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.907 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.908 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:08.908 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.908 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.908 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:08.908 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:08.908 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:08.909 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:08.909 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:08.909 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:08.909 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:08.909 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:08.909 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:08.909 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.910 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.910 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.910 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.910 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:08.910 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:08.910 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.911 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.911 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.911 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:08.911 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.911 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.911 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.911 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:08.911 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.912 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.912 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.912 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.912 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:08.912 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:08.912 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.912 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.912 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.913 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:08.913 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.913 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.913 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:08.913 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:08.913 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:08.913 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:08.914 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:08.914 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:08.914 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:08.914 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:08.914 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:08.914 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.914 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.915 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.915 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:08.915 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:08.915 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.915 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.915 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.916 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:08.916 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.916 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.916 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.916 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:08.916 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.916 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.916 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.916 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.917 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_destination": { 2024-04-25T13:23:08.917 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_destination", 2024-04-25T13:23:08.917 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.917 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.917 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.917 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:08.917 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.918 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.918 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.918 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:23:08.918 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.918 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.918 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.918 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_from_name": { 2024-04-25T13:23:08.919 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_from_name", 2024-04-25T13:23:08.919 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.919 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.919 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.919 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "Ceph", 2024-04-25T13:23:08.919 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.919 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.919 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.920 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email From: name", 2024-04-25T13:23:08.920 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.920 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.920 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.920 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_host": { 2024-04-25T13:23:08.920 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_host", 2024-04-25T13:23:08.920 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.921 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.921 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.921 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:08.921 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.921 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.921 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.921 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP server", 2024-04-25T13:23:08.921 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.922 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.922 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.922 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_password": { 2024-04-25T13:23:08.922 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_password", 2024-04-25T13:23:08.922 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.922 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.922 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.923 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:08.923 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.923 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.923 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.923 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Password to authenticate with", 2024-04-25T13:23:08.923 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.923 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.923 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.924 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_port": { 2024-04-25T13:23:08.924 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_port", 2024-04-25T13:23:08.924 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:08.924 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.924 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.924 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "465", 2024-04-25T13:23:08.924 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.925 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.925 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.925 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP port", 2024-04-25T13:23:08.925 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.925 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.925 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.925 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_sender": { 2024-04-25T13:23:08.925 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_sender", 2024-04-25T13:23:08.926 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.926 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.926 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.926 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:08.926 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.926 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.926 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.926 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:23:08.927 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.927 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.927 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.927 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_ssl": { 2024-04-25T13:23:08.927 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_ssl", 2024-04-25T13:23:08.927 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.927 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.928 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.928 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:08.928 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.928 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.928 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.928 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:23:08.928 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.928 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.929 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.929 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.929 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_user": { 2024-04-25T13:23:08.929 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_user", 2024-04-25T13:23:08.929 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.929 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.929 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.929 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:08.930 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.930 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.930 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.930 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "User to authenticate as", 2024-04-25T13:23:08.930 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.930 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.930 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.930 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:08.931 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:08.931 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.931 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:08.931 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "balancer", 2024-04-25T13:23:08.931 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:08.931 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:08.931 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:08.931 INFO:teuthology.orchestra.run.smithi072.stdout: "active": { 2024-04-25T13:23:08.932 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "active", 2024-04-25T13:23:08.932 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.932 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.932 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.932 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:08.932 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.932 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.932 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.933 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:23:08.933 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.933 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.933 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.933 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.933 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_time": { 2024-04-25T13:23:08.933 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_time", 2024-04-25T13:23:08.933 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.934 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.934 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.934 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0000", 2024-04-25T13:23:08.934 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.934 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.934 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.934 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:23:08.935 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:23:08.935 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.935 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.935 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.935 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_weekday": { 2024-04-25T13:23:08.935 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_weekday", 2024-04-25T13:23:08.936 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:08.936 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.936 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.936 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:08.936 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-25T13:23:08.936 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-25T13:23:08.936 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.937 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:23:08.937 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:23:08.937 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.937 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.937 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.937 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_max_iterations": { 2024-04-25T13:23:08.937 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:23:08.937 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:08.937 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.938 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.938 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "25", 2024-04-25T13:23:08.938 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:08.938 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "250", 2024-04-25T13:23:08.938 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.938 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:23:08.938 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.938 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.939 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.939 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_metrics": { 2024-04-25T13:23:08.939 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_metrics", 2024-04-25T13:23:08.939 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.939 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.939 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.939 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:23:08.940 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.940 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.940 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.940 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:23:08.940 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:23:08.940 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.940 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.941 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.941 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_step": { 2024-04-25T13:23:08.941 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_step", 2024-04-25T13:23:08.941 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:08.941 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.941 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.941 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.5", 2024-04-25T13:23:08.941 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0.001", 2024-04-25T13:23:08.941 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "0.999", 2024-04-25T13:23:08.942 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.942 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:23:08.942 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:23:08.942 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.942 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.942 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.942 INFO:teuthology.orchestra.run.smithi072.stdout: "end_time": { 2024-04-25T13:23:08.942 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_time", 2024-04-25T13:23:08.943 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.943 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.943 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.943 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2359", 2024-04-25T13:23:08.943 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.943 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.943 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.943 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:23:08.944 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:23:08.944 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.944 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.944 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.944 INFO:teuthology.orchestra.run.smithi072.stdout: "end_weekday": { 2024-04-25T13:23:08.944 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_weekday", 2024-04-25T13:23:08.944 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:08.944 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.945 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.945 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:08.945 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-25T13:23:08.945 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-25T13:23:08.945 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.945 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:23:08.945 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:23:08.945 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.945 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.946 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.946 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:08.946 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:08.946 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.946 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.946 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.946 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:08.947 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.947 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.947 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:08.947 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:08.947 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:08.947 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:08.947 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:08.947 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:08.948 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:08.948 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:08.948 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:08.948 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.948 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.948 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.948 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.948 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:08.948 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:08.949 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.949 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.949 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.949 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:08.949 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.949 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.949 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.949 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:08.949 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.950 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.950 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.950 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.950 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:08.950 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:08.950 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.950 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.950 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.951 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:08.951 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.951 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.951 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:08.951 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:08.951 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:08.951 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:08.951 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:08.951 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:08.952 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:08.952 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:08.952 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:08.952 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.952 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.952 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.952 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.952 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:08.953 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:08.953 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.953 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.953 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.953 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:08.953 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.953 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.953 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.954 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:08.954 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.954 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.955 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.955 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.955 INFO:teuthology.orchestra.run.smithi072.stdout: "min_score": { 2024-04-25T13:23:08.955 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_score", 2024-04-25T13:23:08.955 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:08.956 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.956 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.956 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:08.956 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.956 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.956 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.956 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:23:08.956 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.956 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.957 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.957 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.957 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-25T13:23:08.957 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-25T13:23:08.957 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.957 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.957 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.957 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "upmap", 2024-04-25T13:23:08.957 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.958 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.958 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:08.958 INFO:teuthology.orchestra.run.smithi072.stdout: "crush-compat", 2024-04-25T13:23:08.958 INFO:teuthology.orchestra.run.smithi072.stdout: "none", 2024-04-25T13:23:08.958 INFO:teuthology.orchestra.run.smithi072.stdout: "read", 2024-04-25T13:23:08.958 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap", 2024-04-25T13:23:08.958 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap-read" 2024-04-25T13:23:08.959 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:08.959 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Balancer mode", 2024-04-25T13:23:08.959 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.959 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.959 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.959 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.959 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_ids": { 2024-04-25T13:23:08.959 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_ids", 2024-04-25T13:23:08.960 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.960 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.960 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.960 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:08.960 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.960 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.960 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.962 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:08.962 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.962 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.962 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.962 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:23:08.962 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.962 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.963 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.963 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.963 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_deviation": { 2024-04-25T13:23:08.963 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_deviation", 2024-04-25T13:23:08.963 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:08.963 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.963 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.963 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:08.963 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:08.964 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.964 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.964 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:23:08.964 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:23:08.964 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.964 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.964 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.964 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_optimizations": { 2024-04-25T13:23:08.964 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:23:08.965 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:08.965 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.965 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.965 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:08.965 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.965 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.965 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.965 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:23:08.965 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.966 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.966 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.966 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:08.966 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:08.966 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.966 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:08.966 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm", 2024-04-25T13:23:08.966 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:08.967 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:08.967 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:08.967 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_down_multiplier": { 2024-04-25T13:23:08.967 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_down_multiplier", 2024-04-25T13:23:08.967 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:08.967 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.967 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.967 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-25T13:23:08.967 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.968 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.968 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.968 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:23:08.968 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.968 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.968 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.968 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.968 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_refresh_rate": { 2024-04-25T13:23:08.969 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_refresh_rate", 2024-04-25T13:23:08.969 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:08.969 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.969 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.969 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "20", 2024-04-25T13:23:08.969 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.969 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.969 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.969 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:23:08.970 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.970 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.970 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.970 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.970 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_starting_port": { 2024-04-25T13:23:08.970 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_starting_port", 2024-04-25T13:23:08.970 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:08.970 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.970 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.971 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4721", 2024-04-25T13:23:08.971 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.971 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.971 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.971 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:23:08.971 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.971 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.971 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.972 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.972 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_ptrace": { 2024-04-25T13:23:08.972 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_ptrace", 2024-04-25T13:23:08.972 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.972 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.972 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.972 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:08.972 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.973 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.973 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.973 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:23:08.973 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:23:08.973 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.973 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.973 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.973 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_interval": { 2024-04-25T13:23:08.973 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_interval", 2024-04-25T13:23:08.974 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:08.974 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.974 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.974 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:08.974 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.974 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.974 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.974 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:23:08.975 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.975 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.975 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.975 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.975 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:23:08.975 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:23:08.975 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:08.975 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.975 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.976 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.7", 2024-04-25T13:23:08.976 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.976 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.976 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.976 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:23:08.976 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.976 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.976 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.976 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.977 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm_log_destination": { 2024-04-25T13:23:08.977 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm_log_destination", 2024-04-25T13:23:08.977 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.977 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.977 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.977 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:08.977 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.977 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.977 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:08.978 INFO:teuthology.orchestra.run.smithi072.stdout: "file", 2024-04-25T13:23:08.978 INFO:teuthology.orchestra.run.smithi072.stdout: "file,syslog", 2024-04-25T13:23:08.978 INFO:teuthology.orchestra.run.smithi072.stdout: "syslog" 2024-04-25T13:23:08.978 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:08.978 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:23:08.978 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.978 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.982 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.982 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.982 INFO:teuthology.orchestra.run.smithi072.stdout: "cgroups_split": { 2024-04-25T13:23:08.982 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cgroups_split", 2024-04-25T13:23:08.982 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.982 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.982 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.983 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:08.983 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.983 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.983 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.983 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:23:08.983 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.983 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.983 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.983 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.984 INFO:teuthology.orchestra.run.smithi072.stdout: "config_checks_enabled": { 2024-04-25T13:23:08.984 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_checks_enabled", 2024-04-25T13:23:08.984 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.984 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.984 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.984 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:08.984 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.984 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.984 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: "config_dashboard": { 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_dashboard", 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.985 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.986 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:08.986 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.986 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.986 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.986 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:23:08.986 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.986 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.986 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.987 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.987 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_alertmanager": { 2024-04-25T13:23:08.987 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_alertmanager", 2024-04-25T13:23:08.987 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.987 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.987 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.987 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:23:08.987 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.988 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.988 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.988 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:08.988 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.988 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.988 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.988 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.988 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_base": { 2024-04-25T13:23:08.988 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_base", 2024-04-25T13:23:08.989 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.989 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.989 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:08.989 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:23:08.989 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.989 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.989 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.989 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:23:08.989 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.990 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.990 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.990 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.990 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_elasticsearch": { 2024-04-25T13:23:08.990 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:23:08.990 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.990 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.990 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.990 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:23:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "elasticsearch container image", 2024-04-25T13:23:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_grafana": { 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_grafana", 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:08.992 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.993 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.993 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.993 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.993 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_haproxy": { 2024-04-25T13:23:08.993 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_haproxy", 2024-04-25T13:23:08.993 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.993 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.993 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.993 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:23:08.994 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.994 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.994 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.994 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "HAproxy container image", 2024-04-25T13:23:08.994 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.994 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.994 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.995 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.995 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_agent": { 2024-04-25T13:23:08.995 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:23:08.995 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.995 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.995 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.995 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:23:08.995 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.995 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.996 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.996 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:23:08.996 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.996 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.996 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.996 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.996 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_collector": { 2024-04-25T13:23:08.996 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:23:08.996 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.997 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.997 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.997 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:23:08.997 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.997 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.997 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.997 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:23:08.997 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.997 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.998 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.998 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.998 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_query": { 2024-04-25T13:23:08.998 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:23:08.998 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:08.998 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:08.998 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:08.998 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:23:08.999 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:08.999 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:08.999 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:08.999 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger query container image", 2024-04-25T13:23:08.999 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:08.999 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:08.999 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:08.999 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:08.999 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_keepalived": { 2024-04-25T13:23:09.000 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_keepalived", 2024-04-25T13:23:09.000 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.000 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.000 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.000 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:23:09.000 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.000 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.001 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.001 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Keepalived container image", 2024-04-25T13:23:09.001 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.001 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.001 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.001 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.001 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_loki": { 2024-04-25T13:23:09.001 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_loki", 2024-04-25T13:23:09.001 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.002 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.002 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.002 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:23:09.002 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.002 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.002 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.002 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Loki container image", 2024-04-25T13:23:09.002 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.002 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.003 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.003 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.003 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_node_exporter": { 2024-04-25T13:23:09.003 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_node_exporter", 2024-04-25T13:23:09.003 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.003 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.003 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.003 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:23:09.003 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.004 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.004 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.004 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:09.004 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.004 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.004 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.004 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.004 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_nvmeof": { 2024-04-25T13:23:09.005 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_nvmeof", 2024-04-25T13:23:09.005 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.005 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.005 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.005 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:23:09.005 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.005 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.005 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.006 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Nvme-of container image", 2024-04-25T13:23:09.006 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.006 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.006 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.006 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.006 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_prometheus": { 2024-04-25T13:23:09.006 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_prometheus", 2024-04-25T13:23:09.006 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.007 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.008 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.008 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_promtail": { 2024-04-25T13:23:09.008 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_promtail", 2024-04-25T13:23:09.008 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.008 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.008 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.008 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:23:09.008 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.008 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.009 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.009 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Promtail container image", 2024-04-25T13:23:09.009 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.009 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.009 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.009 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.009 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_samba": { 2024-04-25T13:23:09.010 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_samba", 2024-04-25T13:23:09.010 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.010 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.010 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.010 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:23:09.010 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.010 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.010 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.010 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:23:09.011 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.011 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.011 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.011 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.011 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_snmp_gateway": { 2024-04-25T13:23:09.011 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:23:09.011 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.012 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.013 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.013 INFO:teuthology.orchestra.run.smithi072.stdout: "container_init": { 2024-04-25T13:23:09.013 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_init", 2024-04-25T13:23:09.013 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.013 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.013 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.013 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.013 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.014 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.014 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.014 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:23:09.014 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.014 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.014 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.014 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.015 INFO:teuthology.orchestra.run.smithi072.stdout: "daemon_cache_timeout": { 2024-04-25T13:23:09.015 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:23:09.015 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.015 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.015 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.015 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:09.015 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.015 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.015 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:23:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:23:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:23:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.017 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.017 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "900", 2024-04-25T13:23:09.017 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.017 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.017 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.017 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:23:09.017 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.017 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.017 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.018 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.018 INFO:teuthology.orchestra.run.smithi072.stdout: "default_registry": { 2024-04-25T13:23:09.018 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_registry", 2024-04-25T13:23:09.018 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.018 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.018 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.018 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io", 2024-04-25T13:23:09.018 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.019 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.019 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.019 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:23:09.019 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.019 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.019 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.019 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.019 INFO:teuthology.orchestra.run.smithi072.stdout: "device_cache_timeout": { 2024-04-25T13:23:09.019 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_cache_timeout", 2024-04-25T13:23:09.020 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.020 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.020 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.020 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1800", 2024-04-25T13:23:09.020 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.020 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.020 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.020 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:23:09.020 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.021 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.021 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.021 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.021 INFO:teuthology.orchestra.run.smithi072.stdout: "device_enhanced_scan": { 2024-04-25T13:23:09.021 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_enhanced_scan", 2024-04-25T13:23:09.021 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.021 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.021 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.021 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.022 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.022 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.022 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.022 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:23:09.022 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.022 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.022 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.022 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.022 INFO:teuthology.orchestra.run.smithi072.stdout: "facts_cache_timeout": { 2024-04-25T13:23:09.023 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "facts_cache_timeout", 2024-04-25T13:23:09.023 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.023 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.023 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.023 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:09.023 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.023 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.023 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.023 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:23:09.024 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.024 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.024 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.024 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.024 INFO:teuthology.orchestra.run.smithi072.stdout: "grafana_dashboards_path": { 2024-04-25T13:23:09.024 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:23:09.024 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.024 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.024 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.025 INFO:teuthology.orchestra.run.smithi072.stdout: "host_check_interval": { 2024-04-25T13:23:09.026 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "host_check_interval", 2024-04-25T13:23:09.026 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.026 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.026 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.026 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:09.026 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.026 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.026 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.026 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:23:09.027 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.027 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.027 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.027 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.027 INFO:teuthology.orchestra.run.smithi072.stdout: "hw_monitoring": { 2024-04-25T13:23:09.027 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hw_monitoring", 2024-04-25T13:23:09.027 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.027 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.028 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.028 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.028 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.028 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.028 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.028 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:23:09.028 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.028 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.028 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.029 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.029 INFO:teuthology.orchestra.run.smithi072.stdout: "inventory_list_all": { 2024-04-25T13:23:09.029 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "inventory_list_all", 2024-04-25T13:23:09.029 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.029 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.029 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.029 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.029 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.029 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.030 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.030 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:23:09.030 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.030 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.030 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.030 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.030 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.030 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.031 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.031 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.031 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.031 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.031 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.031 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.031 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.031 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.031 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.032 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.032 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.032 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.032 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.032 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.032 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.033 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.033 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.033 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.033 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.033 INFO:teuthology.orchestra.run.smithi072.stdout: "log_refresh_metadata": { 2024-04-25T13:23:09.033 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_refresh_metadata", 2024-04-25T13:23:09.033 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.033 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.033 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.034 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.034 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.034 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.034 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.034 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:23:09.034 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.034 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.035 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.035 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.035 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.035 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.035 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.035 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.035 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.036 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.036 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.036 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.036 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.036 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:23:09.036 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.036 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.036 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.037 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.037 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.037 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.037 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.037 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.037 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.037 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.037 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.038 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.038 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.038 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.038 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.038 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.038 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.038 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.038 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.039 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.039 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.039 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.039 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.039 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.039 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.039 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.039 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.040 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.040 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.040 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.040 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.040 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.040 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.040 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.041 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.041 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.041 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.041 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.041 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.041 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:23:09.041 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:23:09.041 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.041 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.042 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.042 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.042 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.042 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.042 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.042 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:23:09.042 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.042 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.043 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.043 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.043 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:23:09.043 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:23:09.043 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.043 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.043 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.043 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "*", 2024-04-25T13:23:09.043 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.044 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.044 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.044 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:23:09.044 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.044 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.044 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.044 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.044 INFO:teuthology.orchestra.run.smithi072.stdout: "max_count_per_host": { 2024-04-25T13:23:09.044 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_count_per_host", 2024-04-25T13:23:09.045 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.045 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.045 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.045 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.045 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.045 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.045 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.045 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:23:09.046 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.046 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.046 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.046 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.046 INFO:teuthology.orchestra.run.smithi072.stdout: "max_osd_draining_count": { 2024-04-25T13:23:09.046 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_osd_draining_count", 2024-04-25T13:23:09.046 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.046 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.047 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.047 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.047 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.047 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.047 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.047 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:23:09.047 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.047 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.048 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.048 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.048 INFO:teuthology.orchestra.run.smithi072.stdout: "migration_current": { 2024-04-25T13:23:09.048 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "migration_current", 2024-04-25T13:23:09.048 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.048 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.048 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.048 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.048 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.049 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.049 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.049 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "internal - do not modify", 2024-04-25T13:23:09.049 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.049 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.049 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.049 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.049 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-25T13:23:09.049 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-25T13:23:09.050 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.050 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.050 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.050 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "root", 2024-04-25T13:23:09.050 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.050 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.050 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.050 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm-package", 2024-04-25T13:23:09.050 INFO:teuthology.orchestra.run.smithi072.stdout: "root" 2024-04-25T13:23:09.051 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.051 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:23:09.051 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.051 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.051 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.051 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.051 INFO:teuthology.orchestra.run.smithi072.stdout: "oob_default_addr": { 2024-04-25T13:23:09.051 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "oob_default_addr", 2024-04-25T13:23:09.051 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.052 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.052 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.052 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "169.254.1.1", 2024-04-25T13:23:09.052 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.052 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.052 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.052 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:23:09.052 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.053 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.053 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.053 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.053 INFO:teuthology.orchestra.run.smithi072.stdout: "prometheus_alerts_path": { 2024-04-25T13:23:09.053 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:23:09.053 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.053 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.053 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.053 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:23:09.054 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.054 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.054 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.054 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:23:09.054 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.054 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.054 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.054 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.054 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_insecure": { 2024-04-25T13:23:09.055 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_insecure", 2024-04-25T13:23:09.055 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.055 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.055 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.055 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.055 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.055 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.055 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.056 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:23:09.056 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.056 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.056 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.056 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.056 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_password": { 2024-04-25T13:23:09.056 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_password", 2024-04-25T13:23:09.056 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.056 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.058 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_url": { 2024-04-25T13:23:09.058 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_url", 2024-04-25T13:23:09.058 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.058 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.058 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.058 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.058 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.058 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.059 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.059 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:23:09.059 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.059 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.059 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.059 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.059 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_username": { 2024-04-25T13:23:09.059 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_username", 2024-04-25T13:23:09.059 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.060 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.060 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.060 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.060 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.060 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.060 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.060 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:23:09.061 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.061 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.061 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.061 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.061 INFO:teuthology.orchestra.run.smithi072.stdout: "secure_monitoring_stack": { 2024-04-25T13:23:09.061 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:23:09.061 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.061 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:23:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.063 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.063 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.063 INFO:teuthology.orchestra.run.smithi072.stdout: "service_discovery_port": { 2024-04-25T13:23:09.063 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "service_discovery_port", 2024-04-25T13:23:09.063 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.063 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.063 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.063 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8765", 2024-04-25T13:23:09.063 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.064 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.064 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.064 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:23:09.064 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.064 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.064 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.064 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.065 INFO:teuthology.orchestra.run.smithi072.stdout: "ssh_config_file": { 2024-04-25T13:23:09.065 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssh_config_file", 2024-04-25T13:23:09.065 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.065 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.065 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.065 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.065 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.065 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.065 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.066 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:23:09.066 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.066 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.066 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.066 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.066 INFO:teuthology.orchestra.run.smithi072.stdout: "use_agent": { 2024-04-25T13:23:09.066 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_agent", 2024-04-25T13:23:09.066 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:23:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.068 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.068 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.068 INFO:teuthology.orchestra.run.smithi072.stdout: "use_repo_digest": { 2024-04-25T13:23:09.068 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_repo_digest", 2024-04-25T13:23:09.068 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.068 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.068 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.068 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.068 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.069 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.069 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.069 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:23:09.069 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.069 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.069 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.069 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.069 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_failed_host_check": { 2024-04-25T13:23:09.070 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:23:09.070 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.070 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.070 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.070 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.070 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.070 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.070 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.070 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:23:09.071 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.071 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.071 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.071 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.071 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_daemons": { 2024-04-25T13:23:09.071 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:23:09.071 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.071 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.071 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.072 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.072 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.072 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.072 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.072 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:23:09.072 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.072 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.072 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.073 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.073 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_hosts": { 2024-04-25T13:23:09.073 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:23:09.073 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.073 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.073 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.073 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.074 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.074 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.074 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.074 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:23:09.074 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.074 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.074 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.074 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.075 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.075 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.075 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.075 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crash", 2024-04-25T13:23:09.075 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.075 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.075 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.075 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.076 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.076 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.076 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.076 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.076 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.076 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.076 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.076 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.076 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.077 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.077 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.077 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.077 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.077 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.077 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.077 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.077 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.078 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.078 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.078 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.078 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.078 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.078 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.078 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.078 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.079 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.079 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.079 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.079 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.079 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.079 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.079 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.079 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.079 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.080 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.080 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.080 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.080 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.080 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.080 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.080 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.080 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.080 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.081 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.081 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.081 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.081 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.081 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.081 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.081 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.081 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.081 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.083 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.083 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.083 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.083 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.083 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.083 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.083 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.083 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.083 INFO:teuthology.orchestra.run.smithi072.stdout: "retain_interval": { 2024-04-25T13:23:09.084 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retain_interval", 2024-04-25T13:23:09.084 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.084 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.084 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.084 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "31536000", 2024-04-25T13:23:09.084 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.084 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.084 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.084 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:23:09.085 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.085 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.085 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.085 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.085 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_recent_interval": { 2024-04-25T13:23:09.085 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_recent_interval", 2024-04-25T13:23:09.085 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.085 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.086 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.086 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1209600", 2024-04-25T13:23:09.086 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.086 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.086 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.086 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:23:09.086 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.086 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.087 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.087 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.087 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.087 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.087 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.087 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dashboard", 2024-04-25T13:23:09.087 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.087 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.088 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.089 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.089 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.089 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.089 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.089 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.089 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:23:09.089 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:23:09.089 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.089 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.090 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.090 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.090 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.090 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.090 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.090 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.090 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.090 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.090 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.091 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.091 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:23:09.091 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:23:09.091 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.091 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.091 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.091 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.091 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.092 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.092 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.092 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.092 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.092 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.092 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.092 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.092 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:23:09.092 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:23:09.093 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.093 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.093 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.093 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.093 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.093 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.093 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.093 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.093 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.094 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.094 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.094 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.094 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:23:09.094 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:23:09.094 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.094 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.094 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.094 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:23:09.095 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:23:09.096 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.096 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.096 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.096 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.096 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.096 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.096 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.096 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.097 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.097 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.097 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.097 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.097 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:23:09.097 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:23:09.097 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.098 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.098 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.098 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.098 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.098 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.098 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.098 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.099 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.099 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.099 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.099 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.099 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:23:09.099 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:23:09.099 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.100 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.100 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.100 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.100 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.100 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.100 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.100 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.100 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.101 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.101 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.101 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.101 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:23:09.101 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:23:09.101 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.102 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.102 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.102 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.102 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.102 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.103 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.103 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.103 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.103 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.103 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.103 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.103 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:23:09.103 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:23:09.103 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.104 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.104 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.104 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.104 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.104 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.104 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.105 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.106 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.106 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.106 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.106 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.106 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.106 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.106 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.106 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.107 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:23:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:23:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.109 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.109 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.109 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.109 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.109 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.109 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.109 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.109 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.110 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.110 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.110 INFO:teuthology.orchestra.run.smithi072.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:23:09.110 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:23:09.110 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.110 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.110 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.110 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.110 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.111 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.111 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.111 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.111 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.111 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.111 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.111 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.111 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:23:09.111 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:23:09.112 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.112 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.112 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.112 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-25T13:23:09.112 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.112 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.112 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.112 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.113 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.113 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.113 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.113 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.113 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:23:09.113 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:23:09.113 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.113 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.113 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.114 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.114 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.114 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.114 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.114 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.114 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.114 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.114 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.114 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.115 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_URL": { 2024-04-25T13:23:09.115 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:23:09.115 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.115 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.115 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.115 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.115 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.115 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.116 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.117 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.117 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-25T13:23:09.117 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.117 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.117 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.117 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.117 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.117 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.118 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.118 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.118 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:23:09.118 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:23:09.118 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.118 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.118 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.118 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.118 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.119 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.119 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.119 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.119 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.119 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.119 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.119 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.119 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:23:09.120 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:23:09.120 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.120 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.120 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.120 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.120 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.120 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.120 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.120 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.121 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.121 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.121 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.121 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.121 INFO:teuthology.orchestra.run.smithi072.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:23:09.121 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:23:09.121 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.121 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.122 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.122 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.122 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.122 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.122 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.122 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.122 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.122 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.123 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.123 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.123 INFO:teuthology.orchestra.run.smithi072.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:23:09.123 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:23:09.123 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.123 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.123 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.123 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.123 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.124 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.124 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.124 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.124 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.124 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.124 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.124 INFO:teuthology.orchestra.run.smithi072.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:23:09.124 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:23:09.125 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.125 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.125 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.125 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "[]", 2024-04-25T13:23:09.125 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.125 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.125 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.125 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.126 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.126 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.126 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.126 INFO:teuthology.orchestra.run.smithi072.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:23:09.126 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:23:09.126 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.126 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.126 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "{}", 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.127 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:23:09.128 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:23:09.128 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.128 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.128 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.128 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.128 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.128 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.128 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.128 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.129 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.129 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.129 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.129 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.129 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:23:09.129 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:23:09.129 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.129 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.129 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.130 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.130 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.130 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.130 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.130 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.130 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.130 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.130 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.131 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.131 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:23:09.131 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:23:09.131 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.131 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.131 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.131 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.131 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.131 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.132 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.132 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.132 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.132 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.132 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.132 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.132 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:23:09.132 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:23:09.133 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.133 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.133 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.133 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.133 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.133 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.133 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.133 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.133 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.134 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.134 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.134 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.134 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:23:09.134 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:23:09.134 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.134 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.134 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.135 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.135 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.135 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.135 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.135 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.135 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.136 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.136 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.136 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.136 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:23:09.136 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:23:09.136 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.136 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.137 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.137 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.137 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.137 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.137 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.137 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.137 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.137 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.137 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.138 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.138 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:23:09.138 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:23:09.138 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.138 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.138 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.138 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.138 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.138 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.139 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.139 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.139 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.139 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.139 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.139 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.139 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:23:09.139 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:23:09.140 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.140 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.140 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.140 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.140 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.140 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.140 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.141 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.141 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.141 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.141 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.141 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.141 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:23:09.141 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:23:09.141 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.142 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.142 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.142 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.142 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.142 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.142 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.142 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.142 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.143 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.143 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.143 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.143 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:23:09.143 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:23:09.143 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.143 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.143 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.143 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.144 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.144 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.144 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.144 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.144 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.144 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.144 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.144 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.144 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:23:09.145 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:23:09.145 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.145 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.145 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.145 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:23:09.145 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.145 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.145 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.146 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.146 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.146 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.146 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.146 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.146 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:23:09.146 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:23:09.147 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.147 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.147 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.147 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.147 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.147 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.147 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.147 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.148 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.148 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.148 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.148 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.148 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:23:09.148 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:23:09.148 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.148 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.148 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.149 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8", 2024-04-25T13:23:09.149 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.149 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.149 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.149 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.149 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.149 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.149 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.149 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.150 INFO:teuthology.orchestra.run.smithi072.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:23:09.150 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:23:09.150 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.150 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.150 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.150 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "45", 2024-04-25T13:23:09.150 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.150 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.150 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.151 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.151 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.151 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.151 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.151 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.151 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:23:09.151 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:23:09.151 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.151 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.152 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.152 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.152 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.152 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.152 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.152 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.152 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.152 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.152 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.153 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.153 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:23:09.153 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:23:09.153 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.153 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.153 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.153 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-25T13:23:09.153 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.154 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.154 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.154 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.154 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.154 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.154 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.154 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.155 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:23:09.155 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:23:09.155 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.155 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.155 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.155 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.155 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.155 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.155 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.156 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.156 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.156 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.156 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.156 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.156 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:23:09.156 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:23:09.156 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.157 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.158 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.158 INFO:teuthology.orchestra.run.smithi072.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:23:09.158 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:23:09.158 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.158 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.158 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.158 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.158 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.159 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.159 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.159 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.159 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.159 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.159 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.159 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.159 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:23:09.159 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:23:09.160 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.160 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.160 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.160 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:09.160 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.160 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.160 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.161 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.161 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.161 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.161 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.161 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.161 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:23:09.161 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:23:09.161 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.162 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.162 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.162 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.162 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.162 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.162 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.162 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.162 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.162 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.163 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.163 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.163 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:23:09.163 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:23:09.163 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.163 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.163 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.163 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:09.163 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.164 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.164 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.164 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.164 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.164 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.164 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.164 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.164 INFO:teuthology.orchestra.run.smithi072.stdout: "cross_origin_url": { 2024-04-25T13:23:09.164 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cross_origin_url", 2024-04-25T13:23:09.165 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.165 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.165 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.165 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.165 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.165 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.165 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.165 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.165 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.166 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.166 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.166 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.166 INFO:teuthology.orchestra.run.smithi072.stdout: "crt_file": { 2024-04-25T13:23:09.166 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crt_file", 2024-04-25T13:23:09.166 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.166 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.166 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.167 INFO:teuthology.orchestra.run.smithi072.stdout: "debug": { 2024-04-25T13:23:09.168 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "debug", 2024-04-25T13:23:09.168 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.168 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.168 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.168 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.168 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.168 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.168 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.168 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:23:09.169 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.169 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.169 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.169 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.169 INFO:teuthology.orchestra.run.smithi072.stdout: "jwt_token_ttl": { 2024-04-25T13:23:09.169 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "jwt_token_ttl", 2024-04-25T13:23:09.169 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.169 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.169 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.170 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "28800", 2024-04-25T13:23:09.170 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.170 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.170 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.170 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.170 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.170 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.170 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.170 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.171 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-25T13:23:09.171 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-25T13:23:09.171 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.171 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.171 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.171 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.171 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.171 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.171 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.172 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.172 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.172 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.172 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.172 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.172 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.172 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.172 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.172 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.173 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.173 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.173 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.173 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.173 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.173 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.173 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.173 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.173 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.174 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.174 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.174 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.174 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.174 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.174 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.174 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.174 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.174 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.175 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.175 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.175 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.175 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.175 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.175 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.175 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.175 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.175 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.176 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.176 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.176 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.176 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.176 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.176 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.176 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.176 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.176 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.177 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.177 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.177 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.177 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.177 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.177 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.177 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.177 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.177 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.178 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.178 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.178 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.178 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.178 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.178 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.178 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.178 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.179 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.179 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.179 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.179 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.179 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.179 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.179 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.179 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: "motd": { 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "motd", 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.180 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.181 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.181 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.181 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.181 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.181 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "The message of the day", 2024-04-25T13:23:09.181 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.181 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.181 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.181 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.182 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:23:09.182 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:23:09.182 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.182 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.182 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.182 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.182 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.182 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.183 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.183 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.183 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.183 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.183 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.183 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.183 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-25T13:23:09.183 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-25T13:23:09.183 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.184 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.184 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.184 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-25T13:23:09.184 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.184 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.184 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.184 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.184 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.184 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.185 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.185 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.185 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-25T13:23:09.185 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-25T13:23:09.185 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.185 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.185 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.185 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8080", 2024-04-25T13:23:09.185 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.186 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.186 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.186 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.186 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.186 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.186 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.186 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.186 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-25T13:23:09.187 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-25T13:23:09.187 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.187 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.187 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.187 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.187 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.187 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.187 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.188 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.188 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.188 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.188 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.188 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.188 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl_server_port": { 2024-04-25T13:23:09.188 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl_server_port", 2024-04-25T13:23:09.188 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.188 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8443", 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.189 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.190 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-25T13:23:09.190 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-25T13:23:09.190 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.190 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.190 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.190 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "redirect", 2024-04-25T13:23:09.190 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.190 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect" 2024-04-25T13:23:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-25T13:23:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-25T13:23:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-25T13:23:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-25T13:23:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-25T13:23:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: "url_prefix": { 2024-04-25T13:23:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url_prefix", 2024-04-25T13:23:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.194 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.194 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.194 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.194 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.194 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.194 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.194 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.194 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.194 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.195 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.195 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.195 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.195 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.195 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-04-25T13:23:09.195 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.195 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.195 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.195 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_monitoring": { 2024-04-25T13:23:09.196 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_monitoring", 2024-04-25T13:23:09.196 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.196 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.196 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.196 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.196 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.196 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.196 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.196 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "monitor device health metrics", 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.198 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.198 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.198 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.198 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.198 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.198 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.198 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.198 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.199 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.199 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.199 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.199 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.199 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.199 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.199 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.199 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.200 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.200 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.200 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.200 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.200 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.200 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.200 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.200 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.200 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.201 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.201 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.201 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.201 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.201 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.201 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.201 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.201 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.202 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.202 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.202 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.202 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.202 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.202 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.202 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.202 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.203 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.203 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.203 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.203 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.203 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.203 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.203 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.203 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.205 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.205 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.205 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.205 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.205 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.205 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.205 INFO:teuthology.orchestra.run.smithi072.stdout: "mark_out_threshold": { 2024-04-25T13:23:09.205 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mark_out_threshold", 2024-04-25T13:23:09.205 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.206 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.206 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.206 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2419200", 2024-04-25T13:23:09.206 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.206 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.206 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.206 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:23:09.206 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.207 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.207 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.207 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.207 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_name": { 2024-04-25T13:23:09.207 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_name", 2024-04-25T13:23:09.207 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.207 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.207 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.207 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "device_health_metrics", 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: "retention_period": { 2024-04-25T13:23:09.208 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retention_period", 2024-04-25T13:23:09.209 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.209 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.209 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.209 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15552000", 2024-04-25T13:23:09.209 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.209 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.209 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.209 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:23:09.209 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.210 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.210 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.210 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.210 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_frequency": { 2024-04-25T13:23:09.210 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_frequency", 2024-04-25T13:23:09.210 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.210 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.210 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.211 INFO:teuthology.orchestra.run.smithi072.stdout: "self_heal": { 2024-04-25T13:23:09.212 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "self_heal", 2024-04-25T13:23:09.212 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.212 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.212 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.212 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.212 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.212 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.212 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.213 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:23:09.213 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.213 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.213 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.213 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.213 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:09.213 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:09.214 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.214 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.214 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.214 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:09.214 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.214 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.214 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.214 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:23:09.215 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.215 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.215 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.215 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.215 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_threshold": { 2024-04-25T13:23:09.215 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_threshold", 2024-04-25T13:23:09.215 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.215 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.215 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.216 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7257600", 2024-04-25T13:23:09.216 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.216 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.216 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.216 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:23:09.216 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.216 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.216 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.216 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.217 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.217 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.217 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.217 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "diskprediction_local", 2024-04-25T13:23:09.217 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.217 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.217 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.217 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.218 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.218 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.218 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.218 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.218 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.218 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.218 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.218 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.219 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.219 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.219 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.219 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.219 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.219 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.219 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.219 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.219 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.220 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.220 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.220 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.220 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.220 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.220 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.220 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.220 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.220 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.221 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.221 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.221 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.221 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.221 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.221 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.221 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.221 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.222 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.222 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.222 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.222 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.222 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.222 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.222 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.222 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.223 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.223 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.223 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.223 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.223 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.223 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.223 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.223 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.223 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.225 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.225 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.225 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.225 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.225 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.225 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.225 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.225 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.226 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.226 INFO:teuthology.orchestra.run.smithi072.stdout: "predict_interval": { 2024-04-25T13:23:09.226 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predict_interval", 2024-04-25T13:23:09.226 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.226 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.226 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.226 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-25T13:23:09.226 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.226 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.227 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.227 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.227 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.227 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.227 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.227 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.227 INFO:teuthology.orchestra.run.smithi072.stdout: "predictor_model": { 2024-04-25T13:23:09.227 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predictor_model", 2024-04-25T13:23:09.228 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.228 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.228 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.228 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "prophetstor", 2024-04-25T13:23:09.228 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.228 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.228 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.228 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.228 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.229 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.229 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.229 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.229 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:09.229 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:09.229 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.229 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.229 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.230 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:09.230 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.230 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.230 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.230 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.230 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.230 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.230 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.230 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.231 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.231 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.231 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.231 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "influx", 2024-04-25T13:23:09.231 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": false, 2024-04-25T13:23:09.231 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:23:09.231 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.231 INFO:teuthology.orchestra.run.smithi072.stdout: "batch_size": { 2024-04-25T13:23:09.231 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "batch_size", 2024-04-25T13:23:09.232 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.232 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.232 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.232 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5000", 2024-04-25T13:23:09.232 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.232 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.232 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.232 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:23:09.233 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.233 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.233 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.233 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.233 INFO:teuthology.orchestra.run.smithi072.stdout: "database": { 2024-04-25T13:23:09.233 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "database", 2024-04-25T13:23:09.233 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.233 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.233 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.234 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "ceph", 2024-04-25T13:23:09.234 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.234 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.234 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.234 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:23:09.234 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.234 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.234 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.234 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.235 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": { 2024-04-25T13:23:09.235 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hostname", 2024-04-25T13:23:09.235 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.235 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.235 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.235 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.235 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.235 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.236 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.236 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:23:09.236 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.236 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.236 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.236 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:09.237 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:09.237 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.237 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.237 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.237 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "30", 2024-04-25T13:23:09.237 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "5", 2024-04-25T13:23:09.237 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.237 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.237 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:23:09.238 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.238 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.238 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.238 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.238 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.238 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.238 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.239 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.239 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.239 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.239 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.239 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.239 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.239 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.239 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.239 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.240 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.240 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.240 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.240 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.240 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.240 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.240 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.241 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.241 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.241 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.241 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.241 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.241 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.241 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.241 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.241 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.242 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.242 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.242 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.242 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.242 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.242 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.242 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.242 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.243 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.243 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.243 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.243 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.243 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.243 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.243 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.243 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.243 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.244 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.244 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.244 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.244 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.244 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.244 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.244 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.244 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.244 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.246 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.246 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.246 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.246 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.246 INFO:teuthology.orchestra.run.smithi072.stdout: "password": { 2024-04-25T13:23:09.246 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "password", 2024-04-25T13:23:09.246 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.246 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.247 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.247 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.247 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.247 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.247 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.247 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:23:09.247 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.247 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: "port": { 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "port", 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8086", 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.248 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.249 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.249 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server port", 2024-04-25T13:23:09.249 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.249 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.249 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.249 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.249 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-25T13:23:09.249 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-25T13:23:09.249 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.250 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.250 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.250 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "false", 2024-04-25T13:23:09.250 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.250 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.250 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.250 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:23:09.250 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.251 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.251 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.251 INFO:teuthology.orchestra.run.smithi072.stdout: "threads": { 2024-04-25T13:23:09.251 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threads", 2024-04-25T13:23:09.251 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.251 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.251 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.251 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:09.252 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:09.252 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "32", 2024-04-25T13:23:09.252 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.252 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:23:09.252 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.252 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.252 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.252 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.252 INFO:teuthology.orchestra.run.smithi072.stdout: "username": { 2024-04-25T13:23:09.253 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "username", 2024-04-25T13:23:09.253 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.253 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.253 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.253 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.254 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.254 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.254 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.254 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:23:09.254 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.254 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.254 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.255 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.255 INFO:teuthology.orchestra.run.smithi072.stdout: "verify_ssl": { 2024-04-25T13:23:09.255 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "verify_ssl", 2024-04-25T13:23:09.255 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.255 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.255 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.255 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "true", 2024-04-25T13:23:09.255 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.256 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.256 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.256 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:23:09.256 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.256 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.256 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.256 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.256 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "insights", 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.257 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.258 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.258 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.258 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.258 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.258 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.258 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.258 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.258 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.258 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.259 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.259 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.259 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.259 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.259 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.259 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.259 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.259 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.259 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.260 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.260 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.260 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.260 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.260 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.260 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.260 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.260 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.260 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.261 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.261 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.261 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.261 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.261 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.261 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.263 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.263 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.263 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.263 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.263 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.263 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.263 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.263 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.264 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.264 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.264 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.264 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.264 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.264 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.264 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.264 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.264 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.265 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.265 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.265 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.265 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.265 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.265 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.265 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.265 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.265 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.266 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "iostat", 2024-04-25T13:23:09.266 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.266 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.266 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.266 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.266 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.266 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.266 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.266 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.267 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.267 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.267 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.267 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.267 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.267 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.267 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.267 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.267 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.268 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.268 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.268 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.268 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.268 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.268 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.268 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.268 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.268 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.269 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.270 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.270 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.270 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.270 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.270 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.270 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.270 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.270 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.271 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.272 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.272 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.272 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.272 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.272 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.272 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.272 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.272 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.273 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.273 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.273 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.273 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.273 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.273 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.273 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.273 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.273 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.274 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.274 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.274 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.274 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.274 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.274 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "k8sevents", 2024-04-25T13:23:09.274 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.274 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.274 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.275 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_event_retention_days": { 2024-04-25T13:23:09.275 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:23:09.275 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.275 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.275 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.275 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7", 2024-04-25T13:23:09.275 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.275 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.276 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.276 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:23:09.276 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.276 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.276 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.276 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.276 INFO:teuthology.orchestra.run.smithi072.stdout: "config_check_secs": { 2024-04-25T13:23:09.276 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_check_secs", 2024-04-25T13:23:09.277 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.277 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.277 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.277 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.277 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "10", 2024-04-25T13:23:09.277 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.277 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.277 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:23:09.277 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.278 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.278 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.278 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.278 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.278 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.278 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.278 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.278 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.279 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.279 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.279 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.279 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.279 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.279 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.279 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.279 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.280 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.281 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.281 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.281 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.281 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.281 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.281 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.281 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.281 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.281 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.282 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.282 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.282 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.282 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.282 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.282 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.282 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.282 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.282 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.283 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.283 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.283 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.283 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.283 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.283 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.283 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.283 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.284 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.285 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.285 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.285 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.285 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.285 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.285 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.285 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.285 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.285 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.286 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.286 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.286 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.286 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.286 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.286 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "localpool", 2024-04-25T13:23:09.286 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.286 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.286 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.287 INFO:teuthology.orchestra.run.smithi072.stdout: "failure_domain": { 2024-04-25T13:23:09.287 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "failure_domain", 2024-04-25T13:23:09.287 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.287 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.287 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.287 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "host", 2024-04-25T13:23:09.287 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.287 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.287 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.288 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:23:09.288 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:23:09.288 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.288 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.288 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.288 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.288 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.288 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.289 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.289 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.289 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.289 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.289 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.289 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.289 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.289 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.289 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.291 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.291 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.291 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.291 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.291 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.291 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.291 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.291 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.291 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.292 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.292 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.292 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.292 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.292 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.292 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.292 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.292 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.293 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.293 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.293 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.293 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.293 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.293 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.293 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.293 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.294 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.294 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.294 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.294 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.294 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.294 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.294 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.294 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.295 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.295 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.295 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.295 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.295 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.295 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.295 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.296 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.296 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.296 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.296 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.296 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.296 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.296 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.296 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.296 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.297 INFO:teuthology.orchestra.run.smithi072.stdout: "min_size": { 2024-04-25T13:23:09.297 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_size", 2024-04-25T13:23:09.297 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.297 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.297 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.297 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.297 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.297 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.298 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.298 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:23:09.298 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:23:09.298 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.298 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.298 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.298 INFO:teuthology.orchestra.run.smithi072.stdout: "num_rep": { 2024-04-25T13:23:09.298 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "num_rep", 2024-04-25T13:23:09.299 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.299 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.299 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.299 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3", 2024-04-25T13:23:09.299 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.299 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.299 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.300 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:23:09.300 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.300 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.300 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.300 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.300 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_num": { 2024-04-25T13:23:09.300 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_num", 2024-04-25T13:23:09.300 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.301 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.301 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.301 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "128", 2024-04-25T13:23:09.301 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.301 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.301 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.301 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:23:09.301 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.302 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.302 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.302 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.302 INFO:teuthology.orchestra.run.smithi072.stdout: "prefix": { 2024-04-25T13:23:09.302 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prefix", 2024-04-25T13:23:09.302 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.302 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.302 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.303 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.303 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.303 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.303 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.303 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:23:09.303 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.303 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.303 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.304 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.304 INFO:teuthology.orchestra.run.smithi072.stdout: "subtree": { 2024-04-25T13:23:09.304 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "subtree", 2024-04-25T13:23:09.304 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.304 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.304 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.304 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "rack", 2024-04-25T13:23:09.305 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.305 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.305 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.305 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:23:09.305 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:23:09.305 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.305 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.305 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.306 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.306 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.306 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.306 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mds_autoscaler", 2024-04-25T13:23:09.306 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.306 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.306 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.306 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.306 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.307 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.307 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.307 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.307 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.307 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.307 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.307 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.307 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.307 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.308 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.308 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.308 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.308 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.308 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.308 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.308 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.308 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.308 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.309 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.309 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.309 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.309 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.309 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.309 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.309 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.309 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.309 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.310 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.310 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.310 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.310 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.310 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.310 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.310 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.311 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.311 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.311 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.311 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.311 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.311 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.311 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.311 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.311 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.312 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.312 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.312 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.312 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.312 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.312 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.312 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.312 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.312 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.313 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.313 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.313 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.313 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.313 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.313 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.313 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.314 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.314 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.314 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.314 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.314 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.314 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.314 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.314 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.315 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.315 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.315 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.315 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.315 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirroring", 2024-04-25T13:23:09.315 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.315 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.315 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.315 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.316 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.316 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.316 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.316 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.316 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.316 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.316 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.316 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.317 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.317 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.317 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.317 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.317 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.317 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.317 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.317 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.317 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.318 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.318 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.318 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.318 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.318 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.318 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.318 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.319 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.319 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.319 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.319 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.319 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.319 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.319 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.319 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.320 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.320 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.320 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.320 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.320 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.320 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.320 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.320 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.320 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.321 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.321 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.321 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.321 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.321 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.321 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.321 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.321 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.322 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.322 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.322 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.322 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.322 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.322 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.322 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.323 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.323 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.323 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.323 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.323 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.323 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.323 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.323 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.323 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.324 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.324 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.324 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.324 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.324 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.324 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.324 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.324 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "nfs", 2024-04-25T13:23:09.325 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.325 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.325 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.325 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.325 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.325 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.325 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.325 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.325 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.326 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.326 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.326 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.326 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.326 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.326 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.326 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.326 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.326 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.327 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.327 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.327 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.327 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.327 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.327 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.327 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.327 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.327 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.328 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.328 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.328 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.328 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.328 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.328 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.328 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.328 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.328 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.330 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.330 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.330 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.330 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.330 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.330 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.330 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.330 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.331 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.331 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.331 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.331 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.331 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.331 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.331 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.331 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.331 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.332 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.332 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.332 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.332 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.332 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.332 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.332 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.333 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.333 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.333 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.333 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.333 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.333 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.333 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.333 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-25T13:23:09.333 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.334 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.334 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.334 INFO:teuthology.orchestra.run.smithi072.stdout: "fail_fs": { 2024-04-25T13:23:09.334 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "fail_fs", 2024-04-25T13:23:09.334 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.334 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.334 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.334 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.335 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.335 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.335 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.335 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:23:09.335 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.335 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.335 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.335 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.336 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.336 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.336 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.336 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.336 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.336 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.336 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.336 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.336 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.337 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.337 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.337 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.337 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.337 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.337 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.337 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.338 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.338 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.338 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.338 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.338 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.338 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.338 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.338 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.338 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.339 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.339 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.339 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.339 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.339 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.339 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.339 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.339 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.339 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.340 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.340 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.340 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.340 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.340 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.340 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.340 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.340 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.340 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.341 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.341 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.341 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.341 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.341 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.341 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.341 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.342 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.342 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.342 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.342 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.342 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.342 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.342 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.342 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.342 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.343 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.343 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.343 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.343 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.343 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.343 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.343 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.343 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.343 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.344 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.344 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.344 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator": { 2024-04-25T13:23:09.344 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-25T13:23:09.344 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.344 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.344 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.344 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.344 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.345 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.345 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.345 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-04-25T13:23:09.345 INFO:teuthology.orchestra.run.smithi072.stdout: "rook", 2024-04-25T13:23:09.345 INFO:teuthology.orchestra.run.smithi072.stdout: "test_orchestrator" 2024-04-25T13:23:09.345 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.345 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Orchestrator backend", 2024-04-25T13:23:09.345 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.345 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.346 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.346 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.346 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.346 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.346 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.346 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_perf_query", 2024-04-25T13:23:09.346 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.346 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.347 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.347 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.347 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.347 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.347 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.347 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.347 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.347 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.348 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.348 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.348 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.348 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.348 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.348 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.348 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.348 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.349 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.349 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.349 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.349 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.349 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.349 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.349 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.349 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.349 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.350 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.350 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.350 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.350 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.350 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.350 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.350 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.350 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.351 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.351 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.351 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.351 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.351 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.351 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.351 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.351 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.351 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.352 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.352 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.352 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.352 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.352 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.352 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.352 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.352 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.352 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.353 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.353 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.353 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.353 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.353 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.353 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.353 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.353 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.354 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.354 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.354 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.354 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.354 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.354 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.354 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.355 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.355 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.355 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.355 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.355 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.355 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.355 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.355 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.355 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_support", 2024-04-25T13:23:09.356 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.356 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.356 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.356 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.356 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.356 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.356 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.356 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.358 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.358 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.358 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.358 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.358 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.358 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.358 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.359 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.359 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.359 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.359 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.359 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.359 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.359 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.359 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.359 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.360 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.360 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.360 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.360 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.360 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.360 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.360 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.360 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.360 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.361 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.361 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.361 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.361 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.361 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.361 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.361 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.362 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.362 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.362 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.362 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.362 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.362 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.362 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.362 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.363 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.363 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.363 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.363 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.363 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.363 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.363 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.363 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.363 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.364 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.364 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.364 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.364 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.364 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.364 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.364 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.364 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.365 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.365 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_autoscaler", 2024-04-25T13:23:09.365 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.365 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.365 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.365 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.365 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.365 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.365 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.366 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.366 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.366 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.366 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.366 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.366 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.366 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.366 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.367 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.367 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.367 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.367 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.367 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.367 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.367 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.367 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.367 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.368 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.368 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.368 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.368 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.368 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.368 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.368 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.368 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.368 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.369 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.369 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.369 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.369 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.369 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.369 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.369 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.369 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.370 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.370 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.370 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.370 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.370 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.370 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.370 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.370 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.370 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.371 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.371 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.371 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.371 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.371 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.371 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.371 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.371 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.371 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.373 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.373 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.373 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.373 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.373 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.373 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:09.373 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:09.373 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.373 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.374 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.374 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:09.374 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.374 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.374 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.374 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.374 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.374 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: "threshold": { 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threshold", 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1.0", 2024-04-25T13:23:09.375 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.376 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.376 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "scaling threshold", 2024-04-25T13:23:09.376 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:23:09.376 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.376 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.376 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.376 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.376 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.376 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.377 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "progress", 2024-04-25T13:23:09.377 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.377 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.377 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.377 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_pg_recovery_event": { 2024-04-25T13:23:09.377 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:23:09.377 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.377 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.377 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.378 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.378 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.378 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.378 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.378 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:23:09.378 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.378 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.378 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.379 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.379 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-25T13:23:09.379 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-25T13:23:09.379 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.379 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.379 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.379 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.379 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.380 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.380 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.380 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.380 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.380 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.380 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.380 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.380 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.380 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.381 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.382 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.382 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.382 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.382 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.382 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.382 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.382 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.382 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.383 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.384 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.384 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.384 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.384 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.384 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.384 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.384 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.384 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.385 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.385 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.385 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.385 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.385 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.385 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.385 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.385 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.385 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.386 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.386 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.386 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.386 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.386 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.386 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.386 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.386 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.389 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.389 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.389 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.389 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.389 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.390 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.390 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.390 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.390 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.390 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.390 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.390 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.390 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: "max_completed_events": { 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_completed_events", 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "50", 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.391 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.392 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.392 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:23:09.392 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.392 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.392 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.392 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.392 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:09.392 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.393 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.394 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.394 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.394 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.394 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.394 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.394 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus", 2024-04-25T13:23:09.394 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.394 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.395 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.395 INFO:teuthology.orchestra.run.smithi072.stdout: "cache": { 2024-04-25T13:23:09.395 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cache", 2024-04-25T13:23:09.395 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.395 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.395 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.395 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.395 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.395 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.396 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.396 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.396 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.396 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.396 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.396 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.396 INFO:teuthology.orchestra.run.smithi072.stdout: "exclude_perf_counters": { 2024-04-25T13:23:09.396 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "exclude_perf_counters", 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:23:09.397 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.398 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.398 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.398 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.398 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.398 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.398 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.398 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.398 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.399 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.399 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.399 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.399 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.399 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.399 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.399 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.399 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.399 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.400 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.400 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.400 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.400 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.400 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.400 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.400 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.400 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.400 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.401 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.401 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.401 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.401 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.401 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.401 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.401 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.401 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.402 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.402 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.402 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.402 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.402 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.402 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.402 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.402 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.402 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.403 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.403 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.403 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.403 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.403 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.403 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.403 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.403 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.404 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.405 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.405 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.405 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.405 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.405 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.405 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.405 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.405 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.406 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.406 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.406 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.406 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools": { 2024-04-25T13:23:09.406 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools", 2024-04-25T13:23:09.406 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.406 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.406 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.407 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.407 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.407 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.407 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.407 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.407 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.407 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.407 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.407 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.408 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:23:09.408 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:23:09.408 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.408 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.408 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.408 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "300", 2024-04-25T13:23:09.408 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.408 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.408 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.409 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.409 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.409 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.409 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.409 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.409 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_interval": { 2024-04-25T13:23:09.409 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_interval", 2024-04-25T13:23:09.409 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:09.409 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.410 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.410 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15.0", 2024-04-25T13:23:09.410 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.410 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.410 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.410 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.410 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.410 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.411 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.411 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.411 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-25T13:23:09.411 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-25T13:23:09.411 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.411 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.411 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.411 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-25T13:23:09.411 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.412 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.412 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.412 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:23:09.412 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.412 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.412 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.412 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.412 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-25T13:23:09.412 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "9283", 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.414 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.414 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.414 INFO:teuthology.orchestra.run.smithi072.stdout: "stale_cache_strategy": { 2024-04-25T13:23:09.414 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stale_cache_strategy", 2024-04-25T13:23:09.414 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.414 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.414 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.414 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "log", 2024-04-25T13:23:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "default", 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "default", 2024-04-25T13:23:09.416 INFO:teuthology.orchestra.run.smithi072.stdout: "error" 2024-04-25T13:23:09.417 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.417 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.417 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.417 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.417 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.417 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.417 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-25T13:23:09.417 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-25T13:23:09.417 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.418 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.418 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.418 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-25T13:23:09.418 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-25T13:23:09.418 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-25T13:23:09.418 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.418 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.418 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.419 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.419 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.419 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.419 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.419 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.419 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.419 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-04-25T13:23:09.419 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.419 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.420 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.421 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.421 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.421 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.421 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.421 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.421 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.421 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.421 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.421 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.422 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.422 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.422 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.422 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.422 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.422 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.422 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.422 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.423 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.423 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.423 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.423 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.423 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.423 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.423 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.423 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.425 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.425 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.425 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.425 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.425 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.425 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.425 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.425 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.425 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.426 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.426 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.426 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.426 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.426 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.426 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.426 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.426 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.427 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.427 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.427 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.427 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.427 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.427 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.427 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.427 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.427 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.428 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.428 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_snap_create": { 2024-04-25T13:23:09.428 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:23:09.428 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.428 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.428 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.428 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.428 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.429 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.429 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.429 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.429 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.429 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.429 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.429 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.429 INFO:teuthology.orchestra.run.smithi072.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:23:09.429 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:23:09.430 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.430 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.430 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.430 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.430 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.430 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.430 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.430 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.431 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.431 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.431 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.431 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.431 INFO:teuthology.orchestra.run.smithi072.stdout: "trash_purge_schedule": { 2024-04-25T13:23:09.431 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "trash_purge_schedule", 2024-04-25T13:23:09.431 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.432 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.432 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.432 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.432 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.432 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.432 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.432 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.432 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.433 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.433 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.433 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.433 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.433 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.433 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.433 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "restful", 2024-04-25T13:23:09.433 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.434 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.434 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.434 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_auth": { 2024-04-25T13:23:09.434 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_auth", 2024-04-25T13:23:09.434 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.434 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.434 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.434 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.434 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.435 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.435 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.435 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.435 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.435 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.435 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.435 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.435 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-25T13:23:09.436 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-25T13:23:09.436 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.436 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.436 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.436 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.436 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.436 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.436 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.436 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.437 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.437 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.437 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.437 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.437 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.437 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.437 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.437 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.437 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.438 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.438 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.438 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.438 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.438 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.438 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.438 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.439 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.439 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.439 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.439 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.439 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.439 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.439 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.439 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.439 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.440 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.440 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.440 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.440 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.440 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.440 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.440 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.440 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.440 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.441 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.441 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.441 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.441 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.441 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.441 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.441 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.441 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.441 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.442 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.442 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.442 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.442 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.442 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.442 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.442 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.442 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.443 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.444 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.444 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.444 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.444 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.444 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.444 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.444 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.444 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.444 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.445 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.445 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.445 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.445 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.445 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-25T13:23:09.445 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-25T13:23:09.445 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.445 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.446 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.446 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.446 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.446 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.446 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.446 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.446 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.446 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.446 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.447 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.447 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-25T13:23:09.447 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-25T13:23:09.447 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.447 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.447 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.447 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.447 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.448 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.448 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.448 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.448 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.448 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.448 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.448 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.448 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.448 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.449 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.449 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rgw", 2024-04-25T13:23:09.449 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.449 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.449 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.449 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.449 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.449 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.449 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.451 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.452 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.452 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.452 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.452 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.452 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.452 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.452 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.452 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.452 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.453 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.453 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.453 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.453 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.453 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.453 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.453 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.453 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.453 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.454 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.454 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.454 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.454 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.454 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.454 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.454 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.455 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.455 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.455 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.455 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.455 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.455 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.455 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.456 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.456 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.456 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.456 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.456 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.456 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.456 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.456 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.457 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.457 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.457 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.457 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.457 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.457 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.457 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.457 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.457 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.458 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.458 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rook", 2024-04-25T13:23:09.458 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.458 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.458 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.458 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.458 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.458 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.459 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.459 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.459 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.459 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.459 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.459 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.459 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.459 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.459 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.460 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.460 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.460 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.460 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.460 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.460 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.460 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.460 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.460 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.461 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.461 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.461 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.461 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.461 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.461 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.461 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.461 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.462 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.462 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.462 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.462 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.462 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.462 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.462 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.463 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.463 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.463 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.463 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.463 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.463 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.463 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.463 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.463 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.464 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.464 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.464 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.464 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.464 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.464 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.464 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.464 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.464 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.465 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.465 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.465 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.465 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.465 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.465 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.465 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.465 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.465 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_class": { 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "storage_class", 2024-04-25T13:23:09.466 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.467 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.467 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.467 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "local", 2024-04-25T13:23:09.467 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.467 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.467 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.467 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:23:09.467 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "selftest", 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.468 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.469 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.469 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.469 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.469 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.469 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.469 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.469 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.469 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.470 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.470 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.470 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.470 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.470 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.470 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.470 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.470 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.471 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.471 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.471 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.471 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.471 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.471 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.471 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.471 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.472 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.473 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.473 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.473 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.473 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.473 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.473 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.473 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.473 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.473 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.474 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.474 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.474 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.474 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.474 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.474 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.474 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.474 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.475 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.475 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.475 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.475 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.475 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.475 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.475 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.475 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.475 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.476 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.476 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.476 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.476 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.476 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.476 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.476 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.476 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.476 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.477 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.477 INFO:teuthology.orchestra.run.smithi072.stdout: "roption1": { 2024-04-25T13:23:09.477 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption1", 2024-04-25T13:23:09.477 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.477 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.477 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.477 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.477 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.478 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.478 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.478 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.478 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.478 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.478 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.478 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.478 INFO:teuthology.orchestra.run.smithi072.stdout: "roption2": { 2024-04-25T13:23:09.478 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption2", 2024-04-25T13:23:09.479 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.479 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.479 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.479 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "xyz", 2024-04-25T13:23:09.479 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.479 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.479 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.479 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.480 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.480 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.480 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.480 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.480 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption1": { 2024-04-25T13:23:09.480 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption1", 2024-04-25T13:23:09.480 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.480 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.480 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.481 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption2": { 2024-04-25T13:23:09.482 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption2", 2024-04-25T13:23:09.482 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.482 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.482 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.482 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.482 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.482 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.482 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.483 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.483 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.483 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.483 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.483 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.483 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption3": { 2024-04-25T13:23:09.483 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption3", 2024-04-25T13:23:09.483 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:09.483 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.484 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.485 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption4": { 2024-04-25T13:23:09.485 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption4", 2024-04-25T13:23:09.485 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.485 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.485 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.485 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.485 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.485 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.486 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.486 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.486 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.486 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.486 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.486 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.486 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption5": { 2024-04-25T13:23:09.486 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption5", 2024-04-25T13:23:09.486 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.487 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.487 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.487 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.487 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.487 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.487 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.487 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.487 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.488 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.488 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.488 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.488 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption6": { 2024-04-25T13:23:09.488 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption6", 2024-04-25T13:23:09.488 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.488 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.488 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.488 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption7": { 2024-04-25T13:23:09.489 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption7", 2024-04-25T13:23:09.490 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.490 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.490 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.490 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.490 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:09.490 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "42", 2024-04-25T13:23:09.490 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.490 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.491 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.491 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.491 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.491 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.491 INFO:teuthology.orchestra.run.smithi072.stdout: "testkey": { 2024-04-25T13:23:09.491 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testkey", 2024-04-25T13:23:09.491 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.491 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.491 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.492 INFO:teuthology.orchestra.run.smithi072.stdout: "testlkey": { 2024-04-25T13:23:09.493 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testlkey", 2024-04-25T13:23:09.493 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.493 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.493 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.493 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.493 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.493 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.493 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.493 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: "testnewline": { 2024-04-25T13:23:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testnewline", 2024-04-25T13:23:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.495 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.495 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.495 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.495 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.495 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.495 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.495 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.495 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.495 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.496 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.496 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.496 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.496 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-04-25T13:23:09.496 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.496 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.496 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.496 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_m_granularity": { 2024-04-25T13:23:09.496 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_m_granularity", 2024-04-25T13:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.498 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.498 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.498 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.498 INFO:teuthology.orchestra.run.smithi072.stdout: "dump_on_update": { 2024-04-25T13:23:09.498 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dump_on_update", 2024-04-25T13:23:09.498 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.498 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.498 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.499 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.499 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.499 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.499 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.499 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:23:09.499 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.499 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.499 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.499 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.500 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.500 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.500 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.500 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.500 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.500 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.500 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.500 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.501 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.501 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.501 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.501 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.501 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.501 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.501 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.501 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.502 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.502 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.502 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.502 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.502 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.502 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.502 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.502 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.503 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.503 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.503 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.503 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.503 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.503 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.503 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.503 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.505 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.505 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.505 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.505 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.505 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.505 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.505 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.506 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.506 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.506 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.506 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.506 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.506 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.506 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.506 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.508 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.508 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.508 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.508 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.508 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.508 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.508 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.508 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.508 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stats", 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.509 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.510 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.510 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.510 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.510 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.510 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.510 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.510 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.511 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.511 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.511 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.511 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.511 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.511 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.511 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.511 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.511 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.512 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.512 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.512 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.512 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.512 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.512 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.512 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.512 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.512 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.513 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.513 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.513 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.513 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.513 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.513 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.513 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.513 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.513 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.514 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.514 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.514 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.514 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.514 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.514 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.514 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.514 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.515 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.515 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.515 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.515 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.515 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.515 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.515 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.515 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.515 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.516 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.516 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.516 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.516 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.516 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.516 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.516 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.516 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.516 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.517 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.517 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.517 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.517 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.517 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.517 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.517 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.517 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "status", 2024-04-25T13:23:09.517 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.518 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.518 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.518 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.518 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.518 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.518 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.518 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.518 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.518 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.519 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.519 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.519 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.519 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.519 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.519 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.519 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.519 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.520 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.521 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.521 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.521 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.521 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.521 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.521 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.521 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.521 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.521 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.522 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.522 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.522 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.522 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.522 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.522 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.522 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.522 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.523 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.523 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.523 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.523 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.523 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.523 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.523 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.523 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.523 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.524 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.524 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.524 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.524 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.524 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.524 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.524 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.524 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.524 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.525 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.525 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.525 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.525 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.525 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.525 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.525 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.525 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.526 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.526 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.526 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.526 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.526 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.526 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telegraf", 2024-04-25T13:23:09.526 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.526 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.527 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.527 INFO:teuthology.orchestra.run.smithi072.stdout: "address": { 2024-04-25T13:23:09.527 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "address", 2024-04-25T13:23:09.527 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.527 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.527 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.527 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:23:09.527 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.528 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.528 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.528 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.528 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.528 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.528 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.528 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.528 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:09.528 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:09.529 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.529 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.529 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.529 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15", 2024-04-25T13:23:09.529 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.529 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.529 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.529 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.529 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.530 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.531 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.531 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.531 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.531 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.531 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.531 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.531 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.531 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.531 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.532 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.532 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.532 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.532 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.532 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.532 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.532 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.532 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.533 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.533 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.533 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.533 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.533 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.533 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.533 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.533 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.533 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.534 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.534 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.534 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.534 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.534 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.534 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.534 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.534 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.535 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.535 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.535 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.535 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.535 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.535 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.535 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.535 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.536 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.537 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.537 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.537 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.537 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.537 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.537 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.537 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.537 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.537 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.538 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.538 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.538 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.538 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.538 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.538 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telemetry", 2024-04-25T13:23:09.538 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.538 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_basic": { 2024-04-25T13:23:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_basic", 2024-04-25T13:23:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.540 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.540 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:23:09.540 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.540 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.540 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.540 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.540 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_crash": { 2024-04-25T13:23:09.540 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_crash", 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.542 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.542 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.542 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_device": { 2024-04-25T13:23:09.542 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_device", 2024-04-25T13:23:09.542 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.542 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.542 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.542 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_ident": { 2024-04-25T13:23:09.543 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_ident", 2024-04-25T13:23:09.544 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.544 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.544 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.544 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.544 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.544 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.544 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.544 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:23:09.544 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.545 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.545 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.545 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.545 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_perf": { 2024-04-25T13:23:09.545 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_perf", 2024-04-25T13:23:09.545 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.545 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.545 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.546 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.546 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.546 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.546 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.546 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:23:09.546 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.546 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.546 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.547 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.547 INFO:teuthology.orchestra.run.smithi072.stdout: "contact": { 2024-04-25T13:23:09.547 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "contact", 2024-04-25T13:23:09.547 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.547 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.547 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.547 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.548 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.548 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.548 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.548 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.548 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.548 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.548 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.548 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.548 INFO:teuthology.orchestra.run.smithi072.stdout: "description": { 2024-04-25T13:23:09.549 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "description", 2024-04-25T13:23:09.549 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.549 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.549 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.549 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.549 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.549 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.549 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.550 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.550 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.550 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.550 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.550 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.550 INFO:teuthology.orchestra.run.smithi072.stdout: "device_url": { 2024-04-25T13:23:09.550 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_url", 2024-04-25T13:23:09.550 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.550 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:23:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.552 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.552 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-25T13:23:09.552 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-25T13:23:09.552 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.552 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.552 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.552 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.552 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.552 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.553 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.553 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.553 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.553 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.553 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.553 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.553 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:09.553 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:09.553 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.554 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.554 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.554 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "24", 2024-04-25T13:23:09.554 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "8", 2024-04-25T13:23:09.554 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.554 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.554 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.554 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: "last_opt_revision": { 2024-04-25T13:23:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "last_opt_revision", 2024-04-25T13:23:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1", 2024-04-25T13:23:09.556 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.556 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.556 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.556 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.556 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.556 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.556 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.556 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.557 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard": { 2024-04-25T13:23:09.557 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard", 2024-04-25T13:23:09.557 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.557 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.557 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.557 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.557 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.557 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.558 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.558 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.558 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.558 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.558 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.558 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.558 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard_description": { 2024-04-25T13:23:09.558 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard_description", 2024-04-25T13:23:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.560 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.560 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.560 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.560 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.560 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.560 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.560 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.560 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.561 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.562 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.562 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.562 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.562 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.562 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.562 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.563 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.564 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.564 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.564 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.564 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.564 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.564 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.564 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.564 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.564 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.565 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.565 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.565 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.565 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.565 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.565 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.565 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.566 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.566 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.566 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.566 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.566 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.566 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.566 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.566 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.567 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.567 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.567 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.567 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.567 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.567 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.567 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.567 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.567 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.568 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.568 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.568 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.568 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.568 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.568 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.568 INFO:teuthology.orchestra.run.smithi072.stdout: "organization": { 2024-04-25T13:23:09.568 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "organization", 2024-04-25T13:23:09.568 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.569 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.569 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.569 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.569 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.569 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.569 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.569 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.569 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.569 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.570 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.570 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.570 INFO:teuthology.orchestra.run.smithi072.stdout: "proxy": { 2024-04-25T13:23:09.570 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "proxy", 2024-04-25T13:23:09.570 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.570 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.570 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.570 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.570 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.571 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.571 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.571 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.571 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.571 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.571 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.571 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.571 INFO:teuthology.orchestra.run.smithi072.stdout: "url": { 2024-04-25T13:23:09.571 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url", 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.572 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.573 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.573 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.573 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.573 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.573 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.573 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "test_orchestrator", 2024-04-25T13:23:09.573 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.573 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.573 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.574 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.574 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.574 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.574 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.574 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.574 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.574 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.576 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.577 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.577 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.577 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.577 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.577 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.577 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.577 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.577 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.577 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.578 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.578 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.578 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.578 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.578 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.578 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.578 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.578 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.579 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.579 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.579 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.579 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.579 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.579 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.579 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.579 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.580 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.580 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.580 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.580 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.580 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.580 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.580 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.580 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.580 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.581 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.581 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.581 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.581 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.581 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.581 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.581 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.581 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.581 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.582 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.582 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.582 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.582 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.582 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-04-25T13:23:09.582 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.582 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.582 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.582 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.583 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.583 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.583 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.583 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.583 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.583 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.583 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.583 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.583 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.584 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.585 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.585 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.585 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.585 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.585 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.585 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.585 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.585 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.585 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.587 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.587 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.587 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.587 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.587 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.587 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.587 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.587 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.589 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.590 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.590 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.590 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.590 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.590 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.590 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_clones": { 2024-04-25T13:23:09.590 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_clones", 2024-04-25T13:23:09.590 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.591 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.591 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.591 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4", 2024-04-25T13:23:09.591 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.591 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.591 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.591 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:23:09.591 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.591 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.592 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.592 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.592 INFO:teuthology.orchestra.run.smithi072.stdout: "periodic_async_work": { 2024-04-25T13:23:09.592 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "periodic_async_work", 2024-04-25T13:23:09.592 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.592 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.592 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.592 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.592 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Periodically check for async work", 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_delay": { 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:23:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.594 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.594 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.594 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:09.594 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.594 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.594 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.594 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:23:09.594 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.595 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.595 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.595 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.595 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:23:09.595 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:23:09.595 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.595 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.595 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.595 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.596 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.596 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.596 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.596 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:23:09.596 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.596 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.596 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.596 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.596 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.597 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.597 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.597 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix", 2024-04-25T13:23:09.597 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.597 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.597 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.597 INFO:teuthology.orchestra.run.smithi072.stdout: "discovery_interval": { 2024-04-25T13:23:09.597 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "discovery_interval", 2024-04-25T13:23:09.598 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:09.598 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.598 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.598 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "100", 2024-04-25T13:23:09.598 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.598 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.598 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.598 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.599 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.599 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.599 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.599 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.599 INFO:teuthology.orchestra.run.smithi072.stdout: "identifier": { 2024-04-25T13:23:09.599 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "identifier", 2024-04-25T13:23:09.599 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.599 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.599 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.600 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.600 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.600 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.600 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.600 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.600 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.600 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.600 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.600 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.601 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:09.601 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:09.601 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.601 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.601 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.601 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:09.601 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.601 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.602 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.602 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.602 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.602 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.602 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.602 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.602 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.602 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.603 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.603 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.603 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.603 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.603 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.603 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.603 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.603 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.603 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.604 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.604 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.604 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.604 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.604 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.604 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.604 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.604 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.604 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.605 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.605 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.605 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.605 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.605 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.605 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.605 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.605 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.605 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.606 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.606 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.606 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.606 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.606 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.606 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.606 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.607 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.607 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.607 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.607 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.607 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.607 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.607 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.607 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.607 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.608 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.609 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.609 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.609 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.609 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.609 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.609 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.609 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.609 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.609 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.610 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.610 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.610 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.610 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.610 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.610 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.610 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_host": { 2024-04-25T13:23:09.610 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_host", 2024-04-25T13:23:09.610 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.611 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.611 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.611 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.611 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.611 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.611 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.611 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.611 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_port": { 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_port", 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10051", 2024-04-25T13:23:09.612 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.613 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.613 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.613 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.613 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.613 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.613 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.613 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.613 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_sender": { 2024-04-25T13:23:09.614 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_sender", 2024-04-25T13:23:09.614 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.614 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.614 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.614 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:23:09.614 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.614 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.614 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.614 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.615 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.615 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.615 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.615 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.615 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.615 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.615 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:09.615 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.615 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.616 INFO:teuthology.orchestra.run.smithi072.stdout: "gid": 24859, 2024-04-25T13:23:09.616 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "z", 2024-04-25T13:23:09.616 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:23:09.616 INFO:teuthology.orchestra.run.smithi072.stdout: "available_modules": [ 2024-04-25T13:23:09.616 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.616 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "alerts", 2024-04-25T13:23:09.616 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.616 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.616 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:23:09.618 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.618 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.618 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.618 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.618 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.618 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.618 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.618 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.619 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.619 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.619 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.619 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.619 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.619 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.619 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.619 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.619 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.620 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.621 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.621 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.621 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.621 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.621 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.621 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.621 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.621 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.621 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.622 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.622 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.622 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.622 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.622 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.622 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.622 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.622 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.624 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.624 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.624 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.624 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.624 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.624 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.624 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.624 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.624 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.625 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.625 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.625 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.625 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.625 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.625 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.625 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.625 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.625 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.626 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.626 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.626 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.626 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_destination": { 2024-04-25T13:23:09.626 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_destination", 2024-04-25T13:23:09.626 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.626 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.626 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.626 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.627 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.627 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.627 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.627 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:23:09.627 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.627 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.627 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.627 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_from_name": { 2024-04-25T13:23:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_from_name", 2024-04-25T13:23:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "Ceph", 2024-04-25T13:23:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.629 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email From: name", 2024-04-25T13:23:09.629 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.629 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.629 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.629 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.629 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_host": { 2024-04-25T13:23:09.629 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_host", 2024-04-25T13:23:09.629 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.629 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.630 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.630 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.630 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.630 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.630 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.630 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP server", 2024-04-25T13:23:09.630 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.630 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_password": { 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_password", 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.632 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.632 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Password to authenticate with", 2024-04-25T13:23:09.632 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.632 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.632 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.632 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.632 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_port": { 2024-04-25T13:23:09.632 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_port", 2024-04-25T13:23:09.633 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.633 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.633 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.633 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "465", 2024-04-25T13:23:09.633 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.633 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.633 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.633 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP port", 2024-04-25T13:23:09.634 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.634 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.634 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.634 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.634 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_sender": { 2024-04-25T13:23:09.634 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_sender", 2024-04-25T13:23:09.634 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.634 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.635 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.635 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.635 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.635 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.635 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.635 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:23:09.635 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.635 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.635 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.636 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.636 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_ssl": { 2024-04-25T13:23:09.636 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_ssl", 2024-04-25T13:23:09.636 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.636 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.636 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.636 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.636 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.636 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.637 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.637 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:23:09.637 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.637 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.637 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.637 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.637 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_user": { 2024-04-25T13:23:09.637 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_user", 2024-04-25T13:23:09.637 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.638 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.638 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.638 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.638 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.638 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.638 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.639 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "User to authenticate as", 2024-04-25T13:23:09.639 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.639 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.639 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.640 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.640 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.640 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.640 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.640 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "balancer", 2024-04-25T13:23:09.640 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.641 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.641 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.641 INFO:teuthology.orchestra.run.smithi072.stdout: "active": { 2024-04-25T13:23:09.641 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "active", 2024-04-25T13:23:09.641 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.641 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.641 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.641 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.641 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.642 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.642 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.642 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:23:09.642 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.642 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.642 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.642 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.642 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_time": { 2024-04-25T13:23:09.643 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_time", 2024-04-25T13:23:09.643 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.643 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.643 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.643 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0000", 2024-04-25T13:23:09.643 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.643 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.643 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.644 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:23:09.644 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:23:09.644 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.644 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.644 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.644 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_weekday": { 2024-04-25T13:23:09.644 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_weekday", 2024-04-25T13:23:09.644 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:09.644 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.645 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.646 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_max_iterations": { 2024-04-25T13:23:09.646 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:23:09.646 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:09.646 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.646 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.646 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "25", 2024-04-25T13:23:09.646 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:09.647 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "250", 2024-04-25T13:23:09.647 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.647 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:23:09.647 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.647 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.647 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.647 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.648 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_metrics": { 2024-04-25T13:23:09.648 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_metrics", 2024-04-25T13:23:09.648 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.648 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.648 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.648 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:23:09.648 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.648 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.649 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.649 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:23:09.649 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:23:09.649 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.649 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.649 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.649 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_step": { 2024-04-25T13:23:09.649 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_step", 2024-04-25T13:23:09.649 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:09.650 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.650 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.650 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.5", 2024-04-25T13:23:09.650 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0.001", 2024-04-25T13:23:09.650 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "0.999", 2024-04-25T13:23:09.650 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.650 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:23:09.650 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:23:09.651 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.651 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.651 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.651 INFO:teuthology.orchestra.run.smithi072.stdout: "end_time": { 2024-04-25T13:23:09.651 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_time", 2024-04-25T13:23:09.651 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.651 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.651 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.651 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2359", 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: "end_weekday": { 2024-04-25T13:23:09.652 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_weekday", 2024-04-25T13:23:09.653 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:09.653 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.653 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.653 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:09.653 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-25T13:23:09.653 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-25T13:23:09.653 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.653 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:23:09.654 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:23:09.654 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.654 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.654 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.654 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.654 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.654 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.654 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.655 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.655 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.655 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.655 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.655 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.655 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.655 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.655 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.655 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.656 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.656 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.656 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.656 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.656 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.656 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.656 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.656 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.657 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.657 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.657 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.657 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.657 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.657 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.657 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.657 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.658 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.658 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.658 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.658 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.658 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.658 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.658 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.658 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.659 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.659 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.659 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.659 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.659 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.659 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.659 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.659 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.659 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.660 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.661 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.661 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.661 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.661 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.661 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.661 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.661 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.661 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.661 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.662 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.662 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.662 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.662 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.662 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.662 INFO:teuthology.orchestra.run.smithi072.stdout: "min_score": { 2024-04-25T13:23:09.662 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_score", 2024-04-25T13:23:09.663 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:09.663 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.663 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.663 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:09.663 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.663 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.663 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.663 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:23:09.664 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.664 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.664 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.664 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.664 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-25T13:23:09.664 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-25T13:23:09.664 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.664 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.664 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "upmap", 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: "crush-compat", 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: "none", 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: "read", 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap", 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap-read" 2024-04-25T13:23:09.665 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.666 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Balancer mode", 2024-04-25T13:23:09.666 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.666 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.666 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.666 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.666 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_ids": { 2024-04-25T13:23:09.666 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_ids", 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.668 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.668 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.668 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:09.668 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:09.668 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.668 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.668 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.668 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:09.668 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.669 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.669 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.669 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:23:09.669 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.669 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.669 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.669 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.669 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_deviation": { 2024-04-25T13:23:09.669 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_deviation", 2024-04-25T13:23:09.670 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.670 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.670 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.670 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:09.670 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:09.670 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.670 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.670 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:23:09.671 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:23:09.671 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.671 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.671 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.671 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_optimizations": { 2024-04-25T13:23:09.671 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:23:09.671 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:09.671 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.671 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.672 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.673 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.673 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.673 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm", 2024-04-25T13:23:09.673 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.673 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.673 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.673 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_down_multiplier": { 2024-04-25T13:23:09.673 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_down_multiplier", 2024-04-25T13:23:09.674 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:09.674 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.674 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.674 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-25T13:23:09.674 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.674 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.674 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.674 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:23:09.675 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.675 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.675 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.675 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.675 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_refresh_rate": { 2024-04-25T13:23:09.675 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_refresh_rate", 2024-04-25T13:23:09.675 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.675 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.675 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.676 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "20", 2024-04-25T13:23:09.676 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.676 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.676 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.676 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:23:09.676 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.676 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.676 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.676 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_starting_port": { 2024-04-25T13:23:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_starting_port", 2024-04-25T13:23:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4721", 2024-04-25T13:23:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.678 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:23:09.678 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.678 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.678 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.678 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.678 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_ptrace": { 2024-04-25T13:23:09.678 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_ptrace", 2024-04-25T13:23:09.678 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.679 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.679 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.679 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.679 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.679 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.679 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.679 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:23:09.680 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:23:09.680 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.680 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.680 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.681 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_interval": { 2024-04-25T13:23:09.681 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_interval", 2024-04-25T13:23:09.681 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.681 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.681 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.681 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:09.681 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.682 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.682 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.682 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:23:09.682 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.682 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.682 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.7", 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.683 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.684 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:23:09.684 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.684 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.684 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.684 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.684 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm_log_destination": { 2024-04-25T13:23:09.684 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm_log_destination", 2024-04-25T13:23:09.684 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.684 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.685 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.685 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.685 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.685 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.685 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.685 INFO:teuthology.orchestra.run.smithi072.stdout: "file", 2024-04-25T13:23:09.685 INFO:teuthology.orchestra.run.smithi072.stdout: "file,syslog", 2024-04-25T13:23:09.685 INFO:teuthology.orchestra.run.smithi072.stdout: "syslog" 2024-04-25T13:23:09.685 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.686 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:23:09.686 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.686 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.686 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.686 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.686 INFO:teuthology.orchestra.run.smithi072.stdout: "cgroups_split": { 2024-04-25T13:23:09.686 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cgroups_split", 2024-04-25T13:23:09.686 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.687 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.687 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.687 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.687 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.687 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.687 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.687 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:23:09.687 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.688 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.688 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.688 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.688 INFO:teuthology.orchestra.run.smithi072.stdout: "config_checks_enabled": { 2024-04-25T13:23:09.688 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_checks_enabled", 2024-04-25T13:23:09.688 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.688 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.688 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.688 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: "config_dashboard": { 2024-04-25T13:23:09.689 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_dashboard", 2024-04-25T13:23:09.690 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.690 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.690 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.690 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.690 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.690 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.690 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.690 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:23:09.691 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.691 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.691 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.691 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.691 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_alertmanager": { 2024-04-25T13:23:09.691 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_alertmanager", 2024-04-25T13:23:09.691 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.691 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.692 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.693 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_base": { 2024-04-25T13:23:09.693 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_base", 2024-04-25T13:23:09.693 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.693 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.693 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.693 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:23:09.693 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.693 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.693 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.694 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:23:09.694 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.694 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.694 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.694 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.694 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_elasticsearch": { 2024-04-25T13:23:09.694 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:23:09.695 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.695 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.695 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.695 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:23:09.695 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.695 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.695 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.695 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "elasticsearch container image", 2024-04-25T13:23:09.695 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.696 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.696 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.696 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.696 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_grafana": { 2024-04-25T13:23:09.696 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_grafana", 2024-04-25T13:23:09.696 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.696 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.696 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.696 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:23:09.697 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.697 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.697 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.697 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:09.697 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.697 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.697 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.697 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.698 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_haproxy": { 2024-04-25T13:23:09.698 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_haproxy", 2024-04-25T13:23:09.698 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.698 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.698 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.698 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:23:09.698 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.699 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.699 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.699 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "HAproxy container image", 2024-04-25T13:23:09.699 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.699 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.699 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.699 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.699 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_agent": { 2024-04-25T13:23:09.699 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:23:09.700 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.700 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.700 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.700 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:23:09.700 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.700 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.700 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.700 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:23:09.701 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.701 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.701 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.701 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.701 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_collector": { 2024-04-25T13:23:09.701 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:23:09.701 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.701 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.701 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.702 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:23:09.702 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.702 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.702 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.702 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:23:09.702 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.702 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_query": { 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.703 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.704 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.704 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger query container image", 2024-04-25T13:23:09.704 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.704 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.704 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.704 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.704 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_keepalived": { 2024-04-25T13:23:09.704 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_keepalived", 2024-04-25T13:23:09.704 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.705 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.705 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.705 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:23:09.705 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.705 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.705 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.705 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Keepalived container image", 2024-04-25T13:23:09.705 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.705 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.706 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.706 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.706 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_loki": { 2024-04-25T13:23:09.706 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_loki", 2024-04-25T13:23:09.706 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.706 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.706 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.706 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:23:09.707 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.707 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.707 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.707 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Loki container image", 2024-04-25T13:23:09.707 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.707 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.707 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.707 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.707 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_node_exporter": { 2024-04-25T13:23:09.708 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_node_exporter", 2024-04-25T13:23:09.708 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.708 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.708 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.708 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:23:09.708 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.708 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.708 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.708 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_nvmeof": { 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_nvmeof", 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.709 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:23:09.710 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.710 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.710 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.710 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Nvme-of container image", 2024-04-25T13:23:09.710 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.710 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.710 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.711 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.711 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_prometheus": { 2024-04-25T13:23:09.711 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_prometheus", 2024-04-25T13:23:09.711 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.711 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.711 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.711 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:23:09.711 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.711 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_promtail": { 2024-04-25T13:23:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_promtail", 2024-04-25T13:23:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:23:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Promtail container image", 2024-04-25T13:23:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.714 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.714 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.714 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_samba": { 2024-04-25T13:23:09.714 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_samba", 2024-04-25T13:23:09.714 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.714 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.714 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.714 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:23:09.715 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.715 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.715 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.715 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:23:09.715 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.715 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.715 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.715 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.716 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_snmp_gateway": { 2024-04-25T13:23:09.716 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:23:09.716 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.716 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.716 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.716 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:23:09.716 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.716 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.716 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.717 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:23:09.717 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.717 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.717 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.717 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.717 INFO:teuthology.orchestra.run.smithi072.stdout: "container_init": { 2024-04-25T13:23:09.717 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_init", 2024-04-25T13:23:09.717 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.717 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.718 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.718 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.718 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.718 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.718 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.718 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:23:09.718 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.718 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.719 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.719 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.719 INFO:teuthology.orchestra.run.smithi072.stdout: "daemon_cache_timeout": { 2024-04-25T13:23:09.719 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:23:09.719 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.719 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.719 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.719 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:09.719 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.720 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.720 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.720 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:23:09.720 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.720 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.720 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.720 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.720 INFO:teuthology.orchestra.run.smithi072.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:23:09.720 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "900", 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.721 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.722 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.722 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.722 INFO:teuthology.orchestra.run.smithi072.stdout: "default_registry": { 2024-04-25T13:23:09.722 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_registry", 2024-04-25T13:23:09.722 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.722 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.722 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.722 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io", 2024-04-25T13:23:09.723 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.723 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.723 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.723 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:23:09.723 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.723 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.723 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.723 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.724 INFO:teuthology.orchestra.run.smithi072.stdout: "device_cache_timeout": { 2024-04-25T13:23:09.724 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_cache_timeout", 2024-04-25T13:23:09.724 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.724 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.724 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.724 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1800", 2024-04-25T13:23:09.724 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.724 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.725 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.725 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:23:09.725 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.725 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.725 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.725 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.725 INFO:teuthology.orchestra.run.smithi072.stdout: "device_enhanced_scan": { 2024-04-25T13:23:09.725 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_enhanced_scan", 2024-04-25T13:23:09.725 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.726 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.726 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.726 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.726 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.726 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.726 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.726 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:23:09.726 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.727 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.727 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.727 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.727 INFO:teuthology.orchestra.run.smithi072.stdout: "facts_cache_timeout": { 2024-04-25T13:23:09.727 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "facts_cache_timeout", 2024-04-25T13:23:09.727 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.727 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.727 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.727 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:09.728 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.728 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.728 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.728 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:23:09.728 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.728 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.728 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.728 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.729 INFO:teuthology.orchestra.run.smithi072.stdout: "grafana_dashboards_path": { 2024-04-25T13:23:09.729 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:23:09.729 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.729 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.729 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.729 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:23:09.729 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.729 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.729 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.730 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:23:09.730 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.730 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.730 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.730 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.730 INFO:teuthology.orchestra.run.smithi072.stdout: "host_check_interval": { 2024-04-25T13:23:09.731 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "host_check_interval", 2024-04-25T13:23:09.731 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.731 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.731 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:23:09.733 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.733 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.733 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.733 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.734 INFO:teuthology.orchestra.run.smithi072.stdout: "hw_monitoring": { 2024-04-25T13:23:09.734 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hw_monitoring", 2024-04-25T13:23:09.734 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.735 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.735 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.735 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.735 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.735 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.736 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.736 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:23:09.736 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.736 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.737 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.737 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.737 INFO:teuthology.orchestra.run.smithi072.stdout: "inventory_list_all": { 2024-04-25T13:23:09.737 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "inventory_list_all", 2024-04-25T13:23:09.737 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.738 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.738 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.738 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.738 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.738 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.739 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.739 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:23:09.739 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.739 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.739 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.739 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.739 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.739 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.740 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.740 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.740 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.740 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.740 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.740 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.740 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.740 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.741 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.741 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.741 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.741 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.742 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.742 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.742 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.742 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.742 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.742 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.742 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.742 INFO:teuthology.orchestra.run.smithi072.stdout: "log_refresh_metadata": { 2024-04-25T13:23:09.743 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_refresh_metadata", 2024-04-25T13:23:09.743 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.743 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.743 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.743 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.743 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.743 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.743 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.744 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:23:09.744 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.744 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.744 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.744 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.744 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.744 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.744 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.745 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.745 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.745 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.745 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.745 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.745 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.745 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:23:09.745 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.745 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.746 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.746 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.746 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.746 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.746 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.746 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.746 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.747 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.747 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.747 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.747 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.747 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.747 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.748 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.748 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.748 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.748 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.748 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.748 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.748 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.748 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.749 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.749 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.749 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.749 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.749 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.749 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.749 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.749 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.749 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.750 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.750 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.750 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.750 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.750 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.750 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.750 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.750 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:23:09.751 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:23:09.751 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.751 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.751 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.751 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.751 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.751 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.751 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.751 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:23:09.752 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.752 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.752 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.752 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.752 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:23:09.752 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:23:09.752 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.752 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.752 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.753 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "*", 2024-04-25T13:23:09.753 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.753 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.753 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.753 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:23:09.753 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.753 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.753 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.753 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.754 INFO:teuthology.orchestra.run.smithi072.stdout: "max_count_per_host": { 2024-04-25T13:23:09.754 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_count_per_host", 2024-04-25T13:23:09.754 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.754 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.754 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.754 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.754 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.754 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.755 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.755 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:23:09.755 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.755 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.755 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.755 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.755 INFO:teuthology.orchestra.run.smithi072.stdout: "max_osd_draining_count": { 2024-04-25T13:23:09.755 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_osd_draining_count", 2024-04-25T13:23:09.756 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.756 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.756 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.756 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.756 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.756 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.756 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.756 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:23:09.756 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.757 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.757 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.757 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.757 INFO:teuthology.orchestra.run.smithi072.stdout: "migration_current": { 2024-04-25T13:23:09.757 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "migration_current", 2024-04-25T13:23:09.757 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.757 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.757 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.757 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.758 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.758 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.758 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.758 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "internal - do not modify", 2024-04-25T13:23:09.758 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.758 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.758 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.758 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.759 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-25T13:23:09.759 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-25T13:23:09.759 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.759 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.759 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.759 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "root", 2024-04-25T13:23:09.759 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.759 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.759 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.760 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm-package", 2024-04-25T13:23:09.760 INFO:teuthology.orchestra.run.smithi072.stdout: "root" 2024-04-25T13:23:09.760 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.760 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:23:09.760 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.760 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.760 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.760 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.760 INFO:teuthology.orchestra.run.smithi072.stdout: "oob_default_addr": { 2024-04-25T13:23:09.761 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "oob_default_addr", 2024-04-25T13:23:09.761 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.761 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.761 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.761 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "169.254.1.1", 2024-04-25T13:23:09.761 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.761 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.761 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.761 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:23:09.762 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.762 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.762 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.762 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.762 INFO:teuthology.orchestra.run.smithi072.stdout: "prometheus_alerts_path": { 2024-04-25T13:23:09.762 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:23:09.762 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.762 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.763 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.763 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:23:09.763 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.763 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.763 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.763 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:23:09.763 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.763 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.764 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.764 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.764 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_insecure": { 2024-04-25T13:23:09.764 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_insecure", 2024-04-25T13:23:09.764 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.764 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.764 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.764 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.764 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:23:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_password": { 2024-04-25T13:23:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_password", 2024-04-25T13:23:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:23:09.767 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.767 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.767 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.767 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.767 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_url": { 2024-04-25T13:23:09.767 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_url", 2024-04-25T13:23:09.767 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.767 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.767 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.768 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.768 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.768 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.768 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.768 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:23:09.768 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.768 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.768 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.768 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_username": { 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_username", 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.769 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:23:09.770 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.770 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.770 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.770 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.770 INFO:teuthology.orchestra.run.smithi072.stdout: "secure_monitoring_stack": { 2024-04-25T13:23:09.770 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:23:09.770 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.770 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.771 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.771 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.771 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.771 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.771 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.771 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:23:09.771 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.771 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.771 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: "service_discovery_port": { 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "service_discovery_port", 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8765", 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.772 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.773 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:23:09.773 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.773 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.773 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.773 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.773 INFO:teuthology.orchestra.run.smithi072.stdout: "ssh_config_file": { 2024-04-25T13:23:09.773 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssh_config_file", 2024-04-25T13:23:09.773 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.773 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.774 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.774 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.774 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.774 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.774 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.774 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:23:09.774 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.774 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.775 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.775 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.775 INFO:teuthology.orchestra.run.smithi072.stdout: "use_agent": { 2024-04-25T13:23:09.775 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_agent", 2024-04-25T13:23:09.775 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.775 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.775 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.775 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.775 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.776 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.776 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.776 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:23:09.776 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.776 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.776 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.776 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.776 INFO:teuthology.orchestra.run.smithi072.stdout: "use_repo_digest": { 2024-04-25T13:23:09.776 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_repo_digest", 2024-04-25T13:23:09.777 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.777 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.777 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.777 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.777 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.777 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.777 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.777 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:23:09.777 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.778 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.778 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.778 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.778 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_failed_host_check": { 2024-04-25T13:23:09.778 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:23:09.778 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.778 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.778 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:23:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.780 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_daemons": { 2024-04-25T13:23:09.780 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:23:09.780 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.780 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.780 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.780 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.780 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.781 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.781 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.781 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:23:09.781 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.781 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.781 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.781 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.781 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_hosts": { 2024-04-25T13:23:09.781 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:23:09.782 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.782 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.782 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.782 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.782 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.782 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.782 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.782 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:23:09.782 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crash", 2024-04-25T13:23:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.784 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.784 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.784 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.784 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.784 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.784 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.784 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.784 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.784 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.785 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.786 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.786 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.786 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.786 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.786 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.786 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.786 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.786 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.787 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.787 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.787 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.787 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.787 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.787 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.787 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.787 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.787 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.788 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.788 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.788 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.788 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.788 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.788 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.788 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.788 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.788 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.789 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.789 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.789 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.789 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.789 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.789 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.789 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.789 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.789 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.790 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.790 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.790 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.790 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.790 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.790 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.790 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.790 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.790 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.791 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.791 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.791 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.791 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.791 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.791 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.791 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.791 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.791 INFO:teuthology.orchestra.run.smithi072.stdout: "retain_interval": { 2024-04-25T13:23:09.792 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retain_interval", 2024-04-25T13:23:09.792 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.792 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.792 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.792 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "31536000", 2024-04-25T13:23:09.792 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.792 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.792 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.792 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_recent_interval": { 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_recent_interval", 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.793 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1209600", 2024-04-25T13:23:09.794 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.794 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.794 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.794 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:23:09.794 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.794 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.794 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.794 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.795 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.795 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.795 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.795 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dashboard", 2024-04-25T13:23:09.795 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.795 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.795 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.795 INFO:teuthology.orchestra.run.smithi072.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:23:09.795 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:23:09.796 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.796 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.796 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.796 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.796 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.796 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.796 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.796 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.796 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.797 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.797 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.797 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.797 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:23:09.797 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:23:09.797 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.797 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.797 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.797 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.798 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.798 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.798 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.798 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.798 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.798 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.798 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.798 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.798 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:23:09.799 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:23:09.799 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.799 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.799 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.799 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.799 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.799 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.799 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.799 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.800 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.801 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.801 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.801 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.801 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.801 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.801 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.801 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.802 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.802 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:23:09.802 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:23:09.802 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.802 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.802 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.802 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.802 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.803 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.803 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.803 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.803 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.803 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.803 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.803 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.803 INFO:teuthology.orchestra.run.smithi072.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.804 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.805 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.805 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.805 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.805 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:23:09.805 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:23:09.805 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.805 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.805 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.805 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.806 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.806 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.806 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.806 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.806 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.806 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.806 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.807 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.807 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:23:09.807 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:23:09.807 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.807 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.807 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.807 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.807 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.808 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.808 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.808 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.808 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.808 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.808 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.808 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.808 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:23:09.808 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:23:09.809 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.809 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.809 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.809 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.809 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.809 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.809 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.809 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.810 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.810 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.810 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.810 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.810 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:23:09.810 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:23:09.810 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.810 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.811 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.811 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.811 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.811 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.811 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.811 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.811 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.811 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.812 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.812 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.812 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:23:09.812 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:23:09.812 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.812 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.812 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.812 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.812 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.813 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.813 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.813 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.813 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.813 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.813 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.813 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.813 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:23:09.813 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:23:09.814 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.814 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.814 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.814 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.814 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.814 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.814 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.814 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.815 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.815 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.815 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.815 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.815 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:23:09.815 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:23:09.815 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.815 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.815 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.816 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.816 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.816 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.816 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.816 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.816 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.816 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.816 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.816 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.817 INFO:teuthology.orchestra.run.smithi072.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:23:09.817 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:23:09.817 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.817 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.817 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.817 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.817 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.817 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.817 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.818 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.818 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.818 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.818 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.818 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.818 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:23:09.818 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:23:09.818 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.818 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.819 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.819 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-25T13:23:09.819 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.819 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.819 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.819 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.819 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.819 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.820 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.821 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.821 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.821 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.821 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.821 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.821 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.821 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_URL": { 2024-04-25T13:23:09.821 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:23:09.821 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.822 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.822 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.822 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.822 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.822 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.822 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.822 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.822 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.822 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.823 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.823 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.823 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:23:09.823 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:23:09.823 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.823 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.823 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.823 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-25T13:23:09.823 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.824 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.824 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.824 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.824 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.824 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.824 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.824 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.825 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:23:09.825 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:23:09.825 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.825 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.825 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.825 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.825 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.825 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.825 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.826 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.826 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.826 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.826 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.826 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.826 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:23:09.826 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:23:09.826 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.826 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.827 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.827 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.827 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.827 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.827 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.827 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.827 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.827 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.827 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:23:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:23:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.829 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.829 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.829 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.829 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.829 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.829 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.829 INFO:teuthology.orchestra.run.smithi072.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:23:09.829 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:23:09.830 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.830 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.830 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.830 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.830 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.830 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.830 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.830 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.830 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.831 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.831 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.831 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.831 INFO:teuthology.orchestra.run.smithi072.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:23:09.831 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:23:09.831 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.831 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.831 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.831 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "[]", 2024-04-25T13:23:09.832 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.832 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.832 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.832 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.832 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.832 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.832 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.832 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.832 INFO:teuthology.orchestra.run.smithi072.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:23:09.833 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:23:09.833 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.833 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.833 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.833 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "{}", 2024-04-25T13:23:09.833 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.833 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.833 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.833 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.834 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.834 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.834 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.834 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.834 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:23:09.834 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:23:09.834 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.834 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.835 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.835 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.835 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.835 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.835 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.835 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.835 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.835 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.836 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.836 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.836 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:23:09.836 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:23:09.836 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.836 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.836 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.836 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.837 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.837 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.837 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.837 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.837 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.837 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.837 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.837 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.837 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:23:09.838 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:23:09.838 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.838 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.838 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.838 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.838 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.838 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.838 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.839 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.839 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.839 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.839 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.839 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.839 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:23:09.839 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:23:09.839 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.839 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.840 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.840 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.840 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.840 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.840 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.840 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.840 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.841 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.841 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.841 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.841 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:23:09.841 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:23:09.841 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.841 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.841 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.842 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.842 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.842 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.842 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.842 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.842 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.842 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.842 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.842 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.843 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:23:09.843 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:23:09.843 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.843 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.843 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.843 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.843 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.843 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.843 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.844 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.844 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.844 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.844 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.844 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.844 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:23:09.844 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:23:09.844 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.844 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.845 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.845 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.845 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.845 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.845 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.845 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.845 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.845 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.846 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.846 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.846 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:23:09.846 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:23:09.846 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.846 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.846 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.846 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.846 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.847 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.847 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.847 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.847 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.847 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.847 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.847 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.847 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:23:09.847 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:23:09.848 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.848 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.848 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.848 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.848 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.848 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.848 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.849 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.849 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.849 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.849 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.849 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.849 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:23:09.849 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:23:09.849 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.849 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.850 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.850 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.850 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.850 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.850 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.850 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.850 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.851 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.851 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.851 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.851 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:23:09.851 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:23:09.851 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.851 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.851 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.852 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:23:09.852 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.852 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.852 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.852 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.852 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.852 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.852 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:23:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:23:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.854 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.854 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.854 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.854 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.854 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.854 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.854 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:23:09.854 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:23:09.854 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.855 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.855 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.855 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8", 2024-04-25T13:23:09.855 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.855 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.855 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.855 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.855 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.856 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.856 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.856 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.856 INFO:teuthology.orchestra.run.smithi072.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:23:09.856 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:23:09.856 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.856 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.856 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.856 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "45", 2024-04-25T13:23:09.857 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.857 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.857 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.857 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.857 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.857 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.857 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.857 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.857 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:23:09.858 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:23:09.858 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.858 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.858 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.858 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.858 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.858 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.858 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.858 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.859 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.859 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.859 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.859 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.859 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:23:09.859 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:23:09.859 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.859 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.859 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.860 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-25T13:23:09.860 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.860 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.860 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.860 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.860 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.860 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.860 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.860 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.861 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:23:09.861 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:23:09.861 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.861 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.861 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.861 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.861 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.861 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.861 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.862 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.862 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.862 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.862 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.862 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.862 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:23:09.862 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:23:09.862 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.862 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.863 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.863 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.863 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.863 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.863 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.863 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.863 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.863 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.864 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.864 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.864 INFO:teuthology.orchestra.run.smithi072.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:23:09.864 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:23:09.864 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.864 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.864 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.864 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.864 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.865 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.865 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.865 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.865 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.865 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.865 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.865 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.865 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:23:09.865 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.867 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.867 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.867 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.867 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:23:09.867 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:23:09.867 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.867 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.867 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.867 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:23:09.868 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:23:09.869 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.869 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.869 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.869 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:09.869 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.869 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.869 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.869 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.869 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.870 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.870 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.870 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.870 INFO:teuthology.orchestra.run.smithi072.stdout: "cross_origin_url": { 2024-04-25T13:23:09.870 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cross_origin_url", 2024-04-25T13:23:09.870 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.870 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.870 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.870 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.871 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.871 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.871 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.871 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.871 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.871 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.871 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.871 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.871 INFO:teuthology.orchestra.run.smithi072.stdout: "crt_file": { 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crt_file", 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.872 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.873 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.873 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.873 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.873 INFO:teuthology.orchestra.run.smithi072.stdout: "debug": { 2024-04-25T13:23:09.873 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "debug", 2024-04-25T13:23:09.873 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.873 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.873 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.874 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.874 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.874 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.874 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.874 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:23:09.874 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.874 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.874 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.874 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.875 INFO:teuthology.orchestra.run.smithi072.stdout: "jwt_token_ttl": { 2024-04-25T13:23:09.875 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "jwt_token_ttl", 2024-04-25T13:23:09.875 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.875 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.875 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.875 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "28800", 2024-04-25T13:23:09.875 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.875 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.875 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.876 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.876 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.876 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.876 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.876 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.876 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-25T13:23:09.876 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-25T13:23:09.876 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.876 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.877 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.877 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.877 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.877 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.877 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.877 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.877 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.877 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.879 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.880 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.880 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.880 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.880 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.880 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.880 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.880 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.880 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.881 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.881 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.881 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.881 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.881 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.881 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.881 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.881 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.881 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.882 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.882 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.882 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.882 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.882 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.882 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.882 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.882 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.882 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.883 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.883 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.883 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.883 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.883 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.883 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.883 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.883 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.883 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.885 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.885 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.885 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.885 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.885 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.885 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.885 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.885 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.885 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.886 INFO:teuthology.orchestra.run.smithi072.stdout: "motd": { 2024-04-25T13:23:09.886 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "motd", 2024-04-25T13:23:09.886 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.886 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.886 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.886 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.886 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.886 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.887 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.887 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "The message of the day", 2024-04-25T13:23:09.887 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.887 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.887 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.887 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.887 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:23:09.887 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:23:09.887 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.888 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.888 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.888 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.888 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.888 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.888 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.888 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.888 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.889 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.889 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.889 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.889 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-25T13:23:09.889 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-25T13:23:09.889 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.889 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.889 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.890 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-25T13:23:09.890 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.890 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.890 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.890 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.890 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.890 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.890 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.890 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.891 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-25T13:23:09.891 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-25T13:23:09.891 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.891 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.891 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.891 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8080", 2024-04-25T13:23:09.891 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.891 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.891 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.892 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.892 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.892 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.892 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.892 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.892 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-25T13:23:09.892 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-25T13:23:09.892 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.892 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.893 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.894 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl_server_port": { 2024-04-25T13:23:09.894 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl_server_port", 2024-04-25T13:23:09.894 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.894 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.894 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.894 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8443", 2024-04-25T13:23:09.894 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-25T13:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "redirect", 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect" 2024-04-25T13:23:09.896 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.897 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.897 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.897 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.897 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.897 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.897 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-25T13:23:09.897 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-25T13:23:09.897 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.897 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.898 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.898 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-25T13:23:09.898 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-25T13:23:09.898 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-25T13:23:09.898 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.898 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.898 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.898 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.898 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.899 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.899 INFO:teuthology.orchestra.run.smithi072.stdout: "url_prefix": { 2024-04-25T13:23:09.899 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url_prefix", 2024-04-25T13:23:09.899 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.899 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.899 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.899 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.899 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.899 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.900 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.900 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.900 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.900 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.900 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.900 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.900 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.900 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.900 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.901 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-04-25T13:23:09.901 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.901 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.901 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.901 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_monitoring": { 2024-04-25T13:23:09.901 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_monitoring", 2024-04-25T13:23:09.901 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.901 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.902 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.902 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.902 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.902 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.902 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.902 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "monitor device health metrics", 2024-04-25T13:23:09.902 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.902 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.903 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.903 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.903 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.903 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.903 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.903 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.903 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.904 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.904 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.904 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.904 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.904 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.904 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.904 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.904 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.904 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.905 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.905 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.905 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.905 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.905 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.905 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.905 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.905 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.905 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.906 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.906 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.906 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.906 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.906 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.906 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.906 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.906 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.907 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.907 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.907 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.907 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.907 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.907 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.907 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.908 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.908 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.908 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.908 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.908 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.908 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.908 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.908 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.910 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.910 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.910 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.910 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.910 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.910 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.910 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.910 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.910 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.911 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.911 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.911 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.911 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.911 INFO:teuthology.orchestra.run.smithi072.stdout: "mark_out_threshold": { 2024-04-25T13:23:09.912 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mark_out_threshold", 2024-04-25T13:23:09.912 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.912 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.912 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.913 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2419200", 2024-04-25T13:23:09.913 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.913 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.913 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.914 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:23:09.914 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.914 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.914 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.915 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_name": { 2024-04-25T13:23:09.915 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_name", 2024-04-25T13:23:09.915 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.916 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.916 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.916 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "device_health_metrics", 2024-04-25T13:23:09.916 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.917 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.917 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.917 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:23:09.917 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.917 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.918 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.918 INFO:teuthology.orchestra.run.smithi072.stdout: "retention_period": { 2024-04-25T13:23:09.918 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retention_period", 2024-04-25T13:23:09.918 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.919 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.919 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.919 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15552000", 2024-04-25T13:23:09.919 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.919 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.920 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.920 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:23:09.920 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.920 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.921 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.921 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_frequency": { 2024-04-25T13:23:09.921 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_frequency", 2024-04-25T13:23:09.921 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.922 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.922 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.922 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-25T13:23:09.922 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.922 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.923 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.923 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:23:09.923 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.923 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.924 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.924 INFO:teuthology.orchestra.run.smithi072.stdout: "self_heal": { 2024-04-25T13:23:09.924 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "self_heal", 2024-04-25T13:23:09.924 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.925 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.925 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.925 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:09.925 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.926 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.926 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.926 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:23:09.926 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.926 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.927 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.927 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:09.927 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:09.927 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.928 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.928 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.928 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:09.928 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.929 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.929 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.929 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:23:09.929 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.929 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.930 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.930 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.930 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_threshold": { 2024-04-25T13:23:09.930 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_threshold", 2024-04-25T13:23:09.931 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.931 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.931 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.931 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7257600", 2024-04-25T13:23:09.931 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.932 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.932 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.932 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:23:09.932 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.932 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.933 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.933 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.933 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.933 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.933 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.934 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "diskprediction_local", 2024-04-25T13:23:09.934 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.934 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.934 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.934 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.934 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.935 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.935 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.935 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.935 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.936 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.936 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.936 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.936 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.937 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.937 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.937 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.938 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.938 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.938 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.938 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.938 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.938 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.938 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.939 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.939 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.939 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.939 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.939 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.939 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.939 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.939 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.940 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.940 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.940 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.940 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.940 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.940 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.940 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.940 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.941 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.941 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.941 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.941 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.941 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.941 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.941 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.941 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.942 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.942 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.942 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.942 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.942 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.942 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.942 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.942 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.943 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.943 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.943 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.943 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.943 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.943 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.943 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.943 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.944 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.944 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.944 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.944 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.944 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.944 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.944 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.944 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.944 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.945 INFO:teuthology.orchestra.run.smithi072.stdout: "predict_interval": { 2024-04-25T13:23:09.945 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predict_interval", 2024-04-25T13:23:09.945 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.945 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.945 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.945 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-25T13:23:09.945 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.945 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.945 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.946 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.946 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.946 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.946 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.946 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.946 INFO:teuthology.orchestra.run.smithi072.stdout: "predictor_model": { 2024-04-25T13:23:09.947 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predictor_model", 2024-04-25T13:23:09.947 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.947 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.947 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.948 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "prophetstor", 2024-04-25T13:23:09.948 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.948 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.948 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.949 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.949 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.949 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.949 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.950 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.950 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:09.950 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:09.950 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.951 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.951 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.951 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:09.951 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.952 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.952 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.952 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.953 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.953 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.953 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.953 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.953 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.953 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.954 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "influx", 2024-04-25T13:23:09.954 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": false, 2024-04-25T13:23:09.954 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:23:09.954 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.954 INFO:teuthology.orchestra.run.smithi072.stdout: "batch_size": { 2024-04-25T13:23:09.954 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "batch_size", 2024-04-25T13:23:09.954 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.955 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.955 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.955 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5000", 2024-04-25T13:23:09.955 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.955 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.955 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.955 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:23:09.955 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.955 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.956 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.956 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.956 INFO:teuthology.orchestra.run.smithi072.stdout: "database": { 2024-04-25T13:23:09.956 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "database", 2024-04-25T13:23:09.956 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.956 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.956 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.956 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "ceph", 2024-04-25T13:23:09.956 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.957 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.957 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.957 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:23:09.957 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.957 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.957 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.957 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.957 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": { 2024-04-25T13:23:09.958 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hostname", 2024-04-25T13:23:09.958 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.958 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.958 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.958 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.958 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.958 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.958 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.959 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:23:09.959 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.959 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.959 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.959 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.959 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:09.959 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:09.959 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:09.960 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.960 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.960 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "30", 2024-04-25T13:23:09.960 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "5", 2024-04-25T13:23:09.960 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.960 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.960 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:23:09.960 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.960 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.961 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.961 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.961 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.961 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.961 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.961 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.961 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.961 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.961 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.962 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.962 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.962 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.962 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.962 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.962 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.962 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.962 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.963 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.963 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.963 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.963 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.963 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.963 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.963 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.963 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.963 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.964 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.964 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.964 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.964 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.964 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.964 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.964 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.965 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.965 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.965 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.965 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.965 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.965 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.965 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.965 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.966 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.966 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.966 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.966 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.966 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.966 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.966 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.966 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.967 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.968 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.968 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.968 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.968 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.968 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.968 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.968 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.968 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.968 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: "password": { 2024-04-25T13:23:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "password", 2024-04-25T13:23:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.970 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.970 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.970 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.970 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.970 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:23:09.970 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.970 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.970 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: "port": { 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "port", 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8086", 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.971 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server port", 2024-04-25T13:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-25T13:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-25T13:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.973 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.973 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "false", 2024-04-25T13:23:09.973 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.973 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.973 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.973 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:23:09.973 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.973 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.973 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.974 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.974 INFO:teuthology.orchestra.run.smithi072.stdout: "threads": { 2024-04-25T13:23:09.974 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threads", 2024-04-25T13:23:09.974 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.974 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.974 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.974 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:09.974 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "32", 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: "username": { 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "username", 2024-04-25T13:23:09.975 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.976 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.976 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.976 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.976 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.976 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.976 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.976 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:23:09.976 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.976 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.977 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.977 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.977 INFO:teuthology.orchestra.run.smithi072.stdout: "verify_ssl": { 2024-04-25T13:23:09.977 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "verify_ssl", 2024-04-25T13:23:09.977 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.977 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.977 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.977 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "true", 2024-04-25T13:23:09.977 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.978 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.978 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.978 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:23:09.978 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.978 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.978 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.978 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.978 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.978 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "insights", 2024-04-25T13:23:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.981 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.981 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.981 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.981 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.981 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.981 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.981 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.981 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.981 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.982 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.982 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.982 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.982 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.982 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.982 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.982 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.982 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.983 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.983 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.983 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.983 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.983 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.983 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.983 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.983 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.983 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.984 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.984 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.984 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.984 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.985 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.985 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.985 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.985 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.985 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.986 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.986 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.986 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.986 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.986 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.987 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.987 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.987 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.987 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.987 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.987 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.987 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.987 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.989 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.989 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.989 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.989 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "iostat", 2024-04-25T13:23:09.989 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.989 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.989 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.989 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:09.989 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.992 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.992 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.992 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:09.992 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:09.992 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.992 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.992 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.992 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.993 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.993 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.993 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.993 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.993 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.993 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.993 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.993 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.993 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:09.994 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:09.994 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:09.994 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.994 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.994 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:09.994 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.994 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.994 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:09.995 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:09.995 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:09.995 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:09.995 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:09.995 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:09.995 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:09.995 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:09.995 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.995 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:09.996 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.997 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.997 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.997 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:09.997 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.997 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:09.997 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:09.997 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.997 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:09.997 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:09.998 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:09.998 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "k8sevents", 2024-04-25T13:23:09.998 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:09.998 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:09.998 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:09.998 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_event_retention_days": { 2024-04-25T13:23:09.998 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:23:09.998 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:09.999 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:09.999 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:09.999 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7", 2024-04-25T13:23:09.999 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:09.999 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:09.999 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:09.999 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:23:09.999 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:09.999 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.000 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.000 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.000 INFO:teuthology.orchestra.run.smithi072.stdout: "config_check_secs": { 2024-04-25T13:23:10.000 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_check_secs", 2024-04-25T13:23:10.000 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.000 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.000 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.000 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.001 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "10", 2024-04-25T13:23:10.001 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.001 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.001 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:23:10.001 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.001 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.001 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.001 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.001 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.002 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.002 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.002 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.002 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.002 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.002 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.002 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.002 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.003 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.003 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.003 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.003 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.003 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.003 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.003 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.003 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.004 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.004 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.004 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.004 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.004 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.004 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.004 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.004 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.004 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.005 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.005 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.005 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.005 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.005 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.005 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.005 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.005 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.005 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.006 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.006 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.006 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.006 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.006 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.006 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.006 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.006 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.007 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.007 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.007 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.007 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.007 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.007 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.007 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.007 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.007 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.008 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.008 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.008 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.008 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.008 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.008 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.008 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.008 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.008 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.009 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.009 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.009 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.009 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.009 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.009 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.009 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.009 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.009 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.010 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.010 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.010 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.010 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "localpool", 2024-04-25T13:23:10.010 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.010 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.010 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.010 INFO:teuthology.orchestra.run.smithi072.stdout: "failure_domain": { 2024-04-25T13:23:10.010 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "failure_domain", 2024-04-25T13:23:10.011 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.011 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.011 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.011 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "host", 2024-04-25T13:23:10.011 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.011 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.011 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.011 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:23:10.011 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:23:10.012 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.012 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.012 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.012 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.012 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.012 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.012 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.012 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.013 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.013 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.013 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.013 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.013 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.013 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.013 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.013 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.013 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.014 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.014 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.014 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.014 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.014 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.014 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.014 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.015 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.015 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.015 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.015 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.015 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.015 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.015 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.016 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.016 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.016 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.016 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.016 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.016 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.016 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.016 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.016 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.017 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.017 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.017 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.017 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.017 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.017 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.017 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.017 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.017 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.018 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.018 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.018 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.018 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.018 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.018 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.018 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.018 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.019 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.019 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.019 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.019 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.019 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.019 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.019 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.019 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.020 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.020 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.020 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.020 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.020 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.020 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.020 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.020 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.020 INFO:teuthology.orchestra.run.smithi072.stdout: "min_size": { 2024-04-25T13:23:10.021 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_size", 2024-04-25T13:23:10.021 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.021 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.021 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.021 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.021 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.021 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.021 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.021 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:23:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:23:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: "num_rep": { 2024-04-25T13:23:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "num_rep", 2024-04-25T13:23:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.023 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3", 2024-04-25T13:23:10.023 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.023 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.023 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.023 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:23:10.023 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.023 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.023 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.024 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.024 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_num": { 2024-04-25T13:23:10.024 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_num", 2024-04-25T13:23:10.024 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.024 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.024 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.024 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "128", 2024-04-25T13:23:10.024 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.025 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.025 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.025 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:23:10.025 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.025 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.025 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.025 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.025 INFO:teuthology.orchestra.run.smithi072.stdout: "prefix": { 2024-04-25T13:23:10.025 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prefix", 2024-04-25T13:23:10.026 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.026 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.026 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.026 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.026 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.026 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.026 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.026 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:23:10.026 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.027 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.027 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.027 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.027 INFO:teuthology.orchestra.run.smithi072.stdout: "subtree": { 2024-04-25T13:23:10.027 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "subtree", 2024-04-25T13:23:10.027 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.027 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.027 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.027 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "rack", 2024-04-25T13:23:10.028 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.028 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.028 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.028 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:23:10.028 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:23:10.028 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.028 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.028 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.028 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.029 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.029 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.029 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mds_autoscaler", 2024-04-25T13:23:10.029 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.029 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.029 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.029 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.029 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.029 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.030 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.030 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.030 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.030 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.030 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.030 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.030 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.030 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.031 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.031 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.031 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.031 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.031 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.031 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.031 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.031 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.031 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.032 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.032 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.032 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.032 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.032 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.032 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.032 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.032 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.033 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.033 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.033 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.033 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.033 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.033 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.033 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.033 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.034 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.034 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.034 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.034 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.034 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.034 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.034 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.035 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.035 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.035 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.035 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.035 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.035 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.035 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.035 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.035 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.036 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.036 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.036 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.036 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.036 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.036 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.036 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.036 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.037 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.037 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.037 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.037 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.037 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.037 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.037 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.038 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.038 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.038 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.038 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.038 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.038 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.038 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirroring", 2024-04-25T13:23:10.038 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.039 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.039 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.039 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.039 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.039 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.039 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.039 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.039 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.040 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.040 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.040 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.040 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.040 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.040 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.040 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.042 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.042 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.042 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.042 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.043 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.043 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.043 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.043 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.043 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.044 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.044 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.044 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.044 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.045 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.045 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.045 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.045 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.045 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.045 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.045 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.046 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.046 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.046 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.046 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.046 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.046 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.046 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.047 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.047 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.047 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.047 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.047 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.047 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.047 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.047 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.047 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.048 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.048 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.048 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.048 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.048 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.048 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.048 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.048 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.049 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.049 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.049 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.049 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.049 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.049 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.049 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.049 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.049 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "nfs", 2024-04-25T13:23:10.050 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.050 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.050 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.050 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.050 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.050 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.050 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.052 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.052 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.052 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.052 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.052 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.052 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.052 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.052 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.053 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.053 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.053 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.053 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.053 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.053 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.053 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.053 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.053 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.054 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.054 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.054 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.054 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.054 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.054 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.054 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.054 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.055 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.055 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.055 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.055 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.055 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.055 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.055 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.055 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.055 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.056 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.056 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.056 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.056 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.056 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.056 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.056 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.056 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.057 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.057 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.057 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.057 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.057 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.057 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.057 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.057 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.057 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.058 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.058 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.058 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.058 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.058 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.058 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.058 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.059 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.059 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.059 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-25T13:23:10.059 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.059 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.059 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.059 INFO:teuthology.orchestra.run.smithi072.stdout: "fail_fs": { 2024-04-25T13:23:10.059 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "fail_fs", 2024-04-25T13:23:10.059 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:23:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.061 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.061 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.061 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.061 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.061 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.061 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.061 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.061 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.061 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.062 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.062 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.062 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.062 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.062 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.062 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.062 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.062 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.063 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.063 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.063 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.063 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.063 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.063 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.063 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.063 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.064 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.064 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.064 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.064 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.064 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.064 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.065 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.065 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.065 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.065 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.065 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.065 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.065 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.066 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.066 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.066 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.066 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.066 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.066 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.066 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.067 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.067 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.067 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.067 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.067 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.067 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.067 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.067 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.067 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.068 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.069 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.069 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.069 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.069 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.069 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.069 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.069 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.069 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.069 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator": { 2024-04-25T13:23:10.070 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-25T13:23:10.070 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.070 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.070 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.070 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.070 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.070 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.070 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.070 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-04-25T13:23:10.071 INFO:teuthology.orchestra.run.smithi072.stdout: "rook", 2024-04-25T13:23:10.071 INFO:teuthology.orchestra.run.smithi072.stdout: "test_orchestrator" 2024-04-25T13:23:10.071 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.071 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Orchestrator backend", 2024-04-25T13:23:10.071 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.071 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.071 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.071 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.071 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_perf_query", 2024-04-25T13:23:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.073 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.073 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.073 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.073 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.073 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.073 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.073 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.073 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.073 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.074 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.074 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.074 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.074 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.074 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.074 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.074 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.074 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.074 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.075 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.075 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.075 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.075 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.075 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.075 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.075 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.076 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.076 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.076 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.076 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.076 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.076 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.077 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.077 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.077 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.077 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.077 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.077 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.077 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.077 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.077 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.078 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.078 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.078 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.078 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.078 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.078 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.079 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.079 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.079 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.079 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.079 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.079 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.079 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.079 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.080 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.080 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.080 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.080 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.080 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.080 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.080 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.080 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.081 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.081 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.081 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.081 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.081 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.081 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.081 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.082 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.082 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_support", 2024-04-25T13:23:10.082 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.082 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.082 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.082 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.082 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.083 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.083 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.083 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.083 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.083 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.083 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.083 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.084 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.084 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.084 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.084 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.084 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.084 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.084 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.085 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.085 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.085 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.085 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.085 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.085 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.085 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.085 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.086 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.086 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.086 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.086 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.086 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.086 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.086 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.087 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.087 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.087 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.087 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.087 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.087 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.087 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.087 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.088 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.088 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.088 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.088 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.088 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.088 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.088 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.089 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.089 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.089 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.089 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.089 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.089 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.089 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.089 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.090 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.090 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.090 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.090 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.090 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.090 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.090 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.091 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.091 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.091 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.091 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.091 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.091 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.091 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.091 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.092 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.092 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.092 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.092 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.092 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_autoscaler", 2024-04-25T13:23:10.092 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.092 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.092 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.093 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.093 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.093 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.093 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.093 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.093 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.093 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.093 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.094 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.094 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.094 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.094 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.094 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.094 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.094 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.094 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.095 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.095 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.095 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.095 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.095 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.095 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.096 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.096 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.096 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.096 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.096 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.096 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.096 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.097 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.097 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.097 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.097 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.097 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.097 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.097 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.097 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.098 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.098 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.098 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.098 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.098 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.098 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.098 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.099 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.099 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.099 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.099 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.099 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.099 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.099 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.100 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.100 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.100 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.100 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.100 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.100 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.100 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.101 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.102 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.102 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.102 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.102 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.102 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:10.102 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:10.102 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.103 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.103 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.103 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:10.103 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.103 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.103 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.103 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.104 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.104 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.104 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.104 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.104 INFO:teuthology.orchestra.run.smithi072.stdout: "threshold": { 2024-04-25T13:23:10.104 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threshold", 2024-04-25T13:23:10.104 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.105 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.105 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.105 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-25T13:23:10.105 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1.0", 2024-04-25T13:23:10.105 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.105 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.105 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "scaling threshold", 2024-04-25T13:23:10.106 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:23:10.106 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.106 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.106 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.106 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.106 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.107 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.107 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "progress", 2024-04-25T13:23:10.107 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.107 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.107 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.107 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_pg_recovery_event": { 2024-04-25T13:23:10.107 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:23:10.107 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.108 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.108 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.108 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.108 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.108 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.108 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.108 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:23:10.108 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.109 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.109 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.109 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.109 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-25T13:23:10.109 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-25T13:23:10.109 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.109 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.110 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.110 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.110 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.110 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.110 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.110 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.110 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.111 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.111 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.111 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.111 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.111 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.111 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.111 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.111 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.112 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.112 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.112 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.112 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.112 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.112 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.112 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.112 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.113 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.113 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.113 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.113 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.113 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.113 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.113 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.114 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.114 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.114 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.114 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.114 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.114 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.114 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.115 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.115 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.115 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.115 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.115 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.115 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.117 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.117 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.117 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.117 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.117 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.117 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.117 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.118 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.118 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.118 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.118 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.118 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.118 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.118 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.119 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.119 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.119 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.119 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.119 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.119 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.119 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.119 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.120 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.120 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.120 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.120 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.120 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.120 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.120 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.120 INFO:teuthology.orchestra.run.smithi072.stdout: "max_completed_events": { 2024-04-25T13:23:10.121 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_completed_events", 2024-04-25T13:23:10.121 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.121 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.121 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.121 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "50", 2024-04-25T13:23:10.121 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.121 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.121 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.122 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:23:10.122 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.122 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.122 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.122 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.122 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:10.122 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:10.123 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.123 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.123 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.123 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:10.123 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.123 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.123 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.123 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:23:10.124 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.124 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.124 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.124 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.124 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.124 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.125 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus", 2024-04-25T13:23:10.125 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.125 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.125 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.125 INFO:teuthology.orchestra.run.smithi072.stdout: "cache": { 2024-04-25T13:23:10.125 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cache", 2024-04-25T13:23:10.125 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.126 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.126 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.126 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.126 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.126 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.126 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.126 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.127 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.127 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.127 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.127 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.127 INFO:teuthology.orchestra.run.smithi072.stdout: "exclude_perf_counters": { 2024-04-25T13:23:10.127 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "exclude_perf_counters", 2024-04-25T13:23:10.127 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.128 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.128 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.128 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.128 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.128 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.128 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.128 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:23:10.129 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:23:10.129 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.129 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.129 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.129 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.129 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.129 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.130 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.130 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.130 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.130 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.130 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.130 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.130 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.131 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.131 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.131 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.131 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.131 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.131 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.131 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.131 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.132 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.132 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.132 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.132 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.132 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.132 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.132 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.133 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.133 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.133 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.133 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.133 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.133 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.133 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.133 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.134 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.134 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.134 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.134 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.134 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.134 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.135 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.135 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.135 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.135 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.135 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.135 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.135 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.135 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.136 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.136 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.136 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.136 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.136 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.136 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.136 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.136 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.137 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.137 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.137 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.137 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.137 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.137 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.137 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.137 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.138 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.138 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.138 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.138 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.138 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.138 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.138 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.139 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools": { 2024-04-25T13:23:10.139 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools", 2024-04-25T13:23:10.139 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.139 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.139 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.139 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.139 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.139 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.140 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.140 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.140 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.140 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.140 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.140 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.140 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:23:10.140 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:23:10.141 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.141 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.141 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.141 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "300", 2024-04-25T13:23:10.141 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.141 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.141 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.142 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.142 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.142 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.142 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.142 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.142 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_interval": { 2024-04-25T13:23:10.142 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_interval", 2024-04-25T13:23:10.142 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.143 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.143 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.143 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15.0", 2024-04-25T13:23:10.143 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.143 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.143 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.143 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.143 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.144 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.144 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.144 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.144 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-25T13:23:10.144 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-25T13:23:10.144 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.144 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.144 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.145 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-25T13:23:10.145 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.145 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.145 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.145 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:23:10.145 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.145 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.145 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.146 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.146 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-25T13:23:10.146 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-25T13:23:10.146 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.146 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.146 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.146 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "9283", 2024-04-25T13:23:10.147 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.147 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.147 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.147 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:23:10.147 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.147 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.147 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.147 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.148 INFO:teuthology.orchestra.run.smithi072.stdout: "stale_cache_strategy": { 2024-04-25T13:23:10.148 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stale_cache_strategy", 2024-04-25T13:23:10.148 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.148 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.148 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.148 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "log", 2024-04-25T13:23:10.148 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.148 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.148 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.149 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.149 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.149 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.149 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.149 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.149 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-25T13:23:10.149 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-25T13:23:10.149 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.150 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.150 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.150 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "default", 2024-04-25T13:23:10.150 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.150 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.150 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.150 INFO:teuthology.orchestra.run.smithi072.stdout: "default", 2024-04-25T13:23:10.150 INFO:teuthology.orchestra.run.smithi072.stdout: "error" 2024-04-25T13:23:10.151 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.151 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.151 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.151 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.151 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.151 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.151 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-25T13:23:10.151 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-25T13:23:10.152 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.152 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.152 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.152 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-25T13:23:10.152 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-25T13:23:10.152 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-25T13:23:10.152 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.152 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.153 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.153 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.153 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.153 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.153 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.153 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.153 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.153 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-04-25T13:23:10.154 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.154 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.154 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.154 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.154 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.154 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.154 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.154 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.155 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.155 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.155 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.155 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.155 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.155 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.155 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.155 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.156 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.156 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.156 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.156 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.156 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.156 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.156 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.156 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.157 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.157 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.157 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.157 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.157 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.157 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.157 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.157 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.159 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.159 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.159 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.159 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.159 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.159 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.159 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.159 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.160 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.160 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.160 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.160 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.160 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.160 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.160 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.160 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.161 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.161 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.161 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.161 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.161 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.161 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.161 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.161 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.162 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.162 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.162 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.162 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.162 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.162 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.162 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.162 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.163 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_snap_create": { 2024-04-25T13:23:10.163 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:23:10.163 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.163 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.163 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.163 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.163 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.163 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.164 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.164 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.164 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.164 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.164 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.164 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.164 INFO:teuthology.orchestra.run.smithi072.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:23:10.165 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:23:10.165 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.165 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.165 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.165 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.165 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.165 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.166 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.166 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.166 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.166 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.166 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.166 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.166 INFO:teuthology.orchestra.run.smithi072.stdout: "trash_purge_schedule": { 2024-04-25T13:23:10.166 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "trash_purge_schedule", 2024-04-25T13:23:10.167 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.167 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.167 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.167 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.167 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.167 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.167 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.167 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.167 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.168 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.168 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.168 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.168 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.168 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.168 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.168 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "restful", 2024-04-25T13:23:10.168 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.169 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.169 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.169 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_auth": { 2024-04-25T13:23:10.169 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_auth", 2024-04-25T13:23:10.169 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.169 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.169 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.169 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.170 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.170 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.170 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.170 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.170 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.170 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.170 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.170 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.171 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-25T13:23:10.171 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-25T13:23:10.171 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.171 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.171 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.171 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.171 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.171 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.171 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.172 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.172 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.172 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.172 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.172 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.172 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.172 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.172 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.173 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.173 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.173 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.173 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.173 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.173 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.173 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.173 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.174 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.174 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.174 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.174 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.174 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.174 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.174 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.174 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.175 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.175 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.175 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.175 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.175 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.175 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.175 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.175 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.176 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.176 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.176 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.176 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.176 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.176 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.176 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.176 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.177 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.177 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.177 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.177 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.177 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.177 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.177 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.177 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.178 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.178 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.178 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.178 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.178 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.178 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.178 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.178 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.179 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.179 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.179 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.179 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.179 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.179 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.179 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.180 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.180 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.180 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.180 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.180 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.180 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.180 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.180 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.180 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.181 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.181 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.181 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.181 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-25T13:23:10.181 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-25T13:23:10.181 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.181 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.181 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.182 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.182 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.182 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.182 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.182 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.182 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.183 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.183 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.183 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.183 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-25T13:23:10.183 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-25T13:23:10.183 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.183 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.183 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.184 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.184 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.184 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.184 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.184 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.184 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.184 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.184 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.185 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.185 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.185 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.185 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.185 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rgw", 2024-04-25T13:23:10.185 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.185 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.186 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.186 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.186 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.186 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.186 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.186 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.186 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.186 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.186 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.187 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.187 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.187 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.187 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.187 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.187 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.187 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.188 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.188 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.188 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.188 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.188 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.188 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.188 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.189 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.189 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.189 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.189 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.189 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.189 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.189 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.189 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.190 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.190 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.190 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.190 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.190 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.190 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.190 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.191 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.191 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.191 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.191 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.191 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.191 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.191 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.191 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.192 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.192 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.192 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.192 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.192 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.192 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.192 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.192 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.192 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.193 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.193 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.193 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.193 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.193 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.193 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.193 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.193 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.194 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.194 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.194 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.194 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.194 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.194 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.194 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.194 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.195 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.195 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.195 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.195 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rook", 2024-04-25T13:23:10.195 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.195 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.195 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.195 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.196 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.196 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.196 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.196 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.196 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.196 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.196 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.197 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.197 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.197 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.197 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.197 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.197 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.197 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.197 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.197 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.198 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.198 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.198 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.198 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.198 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.198 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.198 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.198 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.199 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.199 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.199 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.199 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.199 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.199 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.199 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.199 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.199 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.200 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.200 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.200 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.200 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.200 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.200 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.200 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.200 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.201 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.201 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.201 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.201 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.201 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.201 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.201 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.201 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.202 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.202 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.202 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.202 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.202 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.202 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.202 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.202 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.203 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.203 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.203 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.203 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.203 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.203 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.203 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.203 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.203 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.204 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.204 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.204 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.204 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_class": { 2024-04-25T13:23:10.204 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "storage_class", 2024-04-25T13:23:10.204 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.204 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.205 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.205 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "local", 2024-04-25T13:23:10.205 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.205 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.205 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.205 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:23:10.205 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.205 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.206 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.206 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.206 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.206 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.206 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.206 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "selftest", 2024-04-25T13:23:10.206 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.207 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.207 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.207 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.207 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.207 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.207 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.207 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.207 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.208 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.208 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.208 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.208 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.208 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.208 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.208 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.209 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.209 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.209 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.209 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.209 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.209 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.209 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.209 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.209 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.210 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.210 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.210 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.210 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.210 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.210 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.210 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.211 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.211 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.211 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.211 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.211 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.211 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.211 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.211 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.212 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.212 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.212 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.212 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.212 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.212 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.212 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.213 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.213 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.213 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.213 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.213 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.213 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.213 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.213 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.214 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.214 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.214 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.214 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.214 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.214 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.214 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.214 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.215 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.215 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.215 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.215 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.215 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.215 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.215 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.216 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.216 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.216 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.216 INFO:teuthology.orchestra.run.smithi072.stdout: "roption1": { 2024-04-25T13:23:10.216 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption1", 2024-04-25T13:23:10.216 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.216 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.216 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.217 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.217 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.217 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.217 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.217 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.217 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.217 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.217 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.218 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.218 INFO:teuthology.orchestra.run.smithi072.stdout: "roption2": { 2024-04-25T13:23:10.218 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption2", 2024-04-25T13:23:10.218 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.218 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.218 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.218 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "xyz", 2024-04-25T13:23:10.218 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.219 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.219 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.219 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.219 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.219 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.219 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.219 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.219 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption1": { 2024-04-25T13:23:10.220 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption1", 2024-04-25T13:23:10.220 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.220 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.220 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.220 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.220 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.220 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.220 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.220 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.221 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.221 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.221 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.221 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.221 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption2": { 2024-04-25T13:23:10.221 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption2", 2024-04-25T13:23:10.221 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.222 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.222 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.222 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.222 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.222 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.222 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.222 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.223 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.223 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.223 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.223 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.223 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption3": { 2024-04-25T13:23:10.223 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption3", 2024-04-25T13:23:10.223 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.224 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.224 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.224 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.224 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.224 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.224 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.224 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.225 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.225 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.225 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.225 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.225 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption4": { 2024-04-25T13:23:10.225 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption4", 2024-04-25T13:23:10.225 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.225 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.226 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.226 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.226 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.226 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.226 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.226 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.226 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.226 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.227 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.227 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.227 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption5": { 2024-04-25T13:23:10.227 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption5", 2024-04-25T13:23:10.227 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.227 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.227 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.228 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.228 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.228 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.228 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.228 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.228 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.228 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.228 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.229 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.229 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption6": { 2024-04-25T13:23:10.229 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption6", 2024-04-25T13:23:10.229 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.229 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.229 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.229 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.229 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.229 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.230 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.230 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.230 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.230 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.230 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.230 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.230 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption7": { 2024-04-25T13:23:10.230 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption7", 2024-04-25T13:23:10.231 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.231 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.231 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.231 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.231 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:10.231 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "42", 2024-04-25T13:23:10.231 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.231 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.232 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.232 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.232 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.232 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.232 INFO:teuthology.orchestra.run.smithi072.stdout: "testkey": { 2024-04-25T13:23:10.232 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testkey", 2024-04-25T13:23:10.232 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.232 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.232 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.233 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.233 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.233 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.233 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.233 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.233 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.233 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.233 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.234 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.234 INFO:teuthology.orchestra.run.smithi072.stdout: "testlkey": { 2024-04-25T13:23:10.234 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testlkey", 2024-04-25T13:23:10.234 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.234 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.234 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.234 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.234 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.235 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.235 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.235 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.235 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.235 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.235 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.235 INFO:teuthology.orchestra.run.smithi072.stdout: "testnewline": { 2024-04-25T13:23:10.236 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testnewline", 2024-04-25T13:23:10.236 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.236 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.236 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.236 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.236 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.236 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.237 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.237 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.237 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.237 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.237 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.237 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.237 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.238 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.238 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-04-25T13:23:10.238 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.238 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.238 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.238 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_m_granularity": { 2024-04-25T13:23:10.239 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_m_granularity", 2024-04-25T13:23:10.239 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.239 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.239 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.239 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.239 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.239 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.239 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.240 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:23:10.240 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.240 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.240 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.240 INFO:teuthology.orchestra.run.smithi072.stdout: "dump_on_update": { 2024-04-25T13:23:10.240 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dump_on_update", 2024-04-25T13:23:10.240 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.241 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.241 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.241 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.241 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.241 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.241 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.241 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:23:10.241 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.242 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.242 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.242 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.242 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.242 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.242 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.242 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.243 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.243 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.243 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.243 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.243 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.243 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.243 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.243 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.244 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.244 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.244 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.244 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.244 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.244 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.244 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.244 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.244 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.245 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.245 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.245 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.245 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.245 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.245 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.245 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.246 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.246 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.246 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.246 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.246 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.246 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.246 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.246 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.247 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.247 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.247 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.247 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.247 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.247 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.247 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.247 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.248 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.248 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.248 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.248 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.248 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.248 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.248 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.248 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.249 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.249 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.249 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.249 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.249 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.249 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.249 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.249 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.250 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.250 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.250 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.250 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.250 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.250 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.250 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.250 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.251 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.251 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.251 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.251 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stats", 2024-04-25T13:23:10.251 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.251 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.251 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.252 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.252 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.252 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.252 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.252 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.252 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.252 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.252 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.253 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.253 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.253 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.253 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.253 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.253 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.253 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.253 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.253 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.254 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.254 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.254 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.254 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.254 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.254 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.254 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.254 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.255 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.255 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.255 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.255 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.255 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.255 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.255 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.256 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.256 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.256 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.256 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.256 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.256 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.257 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.257 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.257 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.257 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.257 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.257 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.257 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.257 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.257 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.258 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.258 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.258 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.258 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.258 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.258 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.258 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.258 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.259 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.259 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.259 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.259 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.259 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.259 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.259 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.259 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.259 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.260 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.260 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.260 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.260 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.260 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.260 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.260 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.260 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.261 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "status", 2024-04-25T13:23:10.261 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.261 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.261 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.261 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.261 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.261 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.261 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.262 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.262 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.262 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.262 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.262 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.262 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.262 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.262 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.263 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.263 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.263 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.263 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.263 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.263 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.263 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.264 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.264 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.264 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.264 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.264 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.264 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.264 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.264 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.265 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.265 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.265 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.265 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.265 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.265 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.265 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.266 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.266 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.266 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.266 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.266 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.266 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.266 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.266 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.267 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.267 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.267 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.267 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.267 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.267 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.267 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.267 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.267 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.268 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.268 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.268 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.268 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.268 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.268 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.268 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.268 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.269 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.269 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.269 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.269 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.269 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.269 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.269 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.269 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.269 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.270 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.270 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.270 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.270 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.270 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.270 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telegraf", 2024-04-25T13:23:10.270 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.270 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.271 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.271 INFO:teuthology.orchestra.run.smithi072.stdout: "address": { 2024-04-25T13:23:10.271 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "address", 2024-04-25T13:23:10.271 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.271 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.271 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.271 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:23:10.271 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.272 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.272 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.272 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.272 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.272 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.272 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.272 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.273 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:10.273 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:10.273 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.273 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.273 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.273 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15", 2024-04-25T13:23:10.273 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.274 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.274 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.274 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.274 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.274 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.274 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.274 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.275 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.275 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.275 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.275 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.275 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.275 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.275 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.275 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.276 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.276 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.276 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.276 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.276 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.276 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.276 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.277 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.277 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.277 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.277 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.277 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.277 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.277 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.277 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.278 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.278 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.278 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.278 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.278 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.278 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.278 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.279 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.279 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.279 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.279 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.279 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.279 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.279 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.280 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.280 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.280 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.280 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.280 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.280 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.280 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.280 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.281 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.281 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.281 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.281 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.281 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.281 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.281 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.281 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.282 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.282 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.282 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.282 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.282 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.282 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.282 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.283 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.283 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.283 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.283 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.283 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.283 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.283 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.283 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.284 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.284 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.284 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.284 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.284 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.284 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telemetry", 2024-04-25T13:23:10.284 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.284 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.285 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.285 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_basic": { 2024-04-25T13:23:10.285 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_basic", 2024-04-25T13:23:10.285 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.285 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.285 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.285 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.286 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.286 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.286 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.286 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:23:10.286 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.286 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.286 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.287 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.287 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_crash": { 2024-04-25T13:23:10.287 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_crash", 2024-04-25T13:23:10.287 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.287 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.287 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.287 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.287 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.288 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.288 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.288 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:23:10.288 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.288 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.288 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.288 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.289 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_device": { 2024-04-25T13:23:10.289 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_device", 2024-04-25T13:23:10.289 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.289 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.289 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.289 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.289 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.289 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.290 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.290 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:23:10.290 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.290 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.290 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.290 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.291 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_ident": { 2024-04-25T13:23:10.291 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_ident", 2024-04-25T13:23:10.291 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.291 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.291 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.291 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.291 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.291 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.292 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.292 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:23:10.292 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.292 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.292 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.292 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.292 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_perf": { 2024-04-25T13:23:10.292 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_perf", 2024-04-25T13:23:10.293 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.293 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.293 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.293 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.293 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.293 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.293 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.293 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:23:10.294 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.294 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.294 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.294 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.294 INFO:teuthology.orchestra.run.smithi072.stdout: "contact": { 2024-04-25T13:23:10.294 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "contact", 2024-04-25T13:23:10.294 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.295 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.295 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.295 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.295 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.295 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.295 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.295 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.295 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.296 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.296 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.296 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.296 INFO:teuthology.orchestra.run.smithi072.stdout: "description": { 2024-04-25T13:23:10.296 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "description", 2024-04-25T13:23:10.296 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.296 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.296 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.297 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.297 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.297 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.297 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.297 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.297 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.297 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.297 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.298 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.298 INFO:teuthology.orchestra.run.smithi072.stdout: "device_url": { 2024-04-25T13:23:10.298 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_url", 2024-04-25T13:23:10.298 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.298 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.298 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.298 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:23:10.298 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.299 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.299 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.299 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.299 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.299 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.299 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.299 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.299 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-25T13:23:10.300 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-25T13:23:10.300 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.300 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.300 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.300 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.300 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.300 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.300 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.301 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.301 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.301 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.301 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.301 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.301 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:10.301 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:10.301 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.302 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.302 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.302 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "24", 2024-04-25T13:23:10.302 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "8", 2024-04-25T13:23:10.302 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.302 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.302 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.303 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.303 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.303 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.303 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.303 INFO:teuthology.orchestra.run.smithi072.stdout: "last_opt_revision": { 2024-04-25T13:23:10.303 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "last_opt_revision", 2024-04-25T13:23:10.303 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.303 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.304 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.304 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1", 2024-04-25T13:23:10.304 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.304 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.304 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.304 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.304 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.304 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.305 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.305 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.305 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard": { 2024-04-25T13:23:10.305 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard", 2024-04-25T13:23:10.305 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.305 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.305 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.305 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.306 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.306 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.306 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.306 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.306 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.306 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.306 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.306 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.307 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard_description": { 2024-04-25T13:23:10.307 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard_description", 2024-04-25T13:23:10.307 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.307 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.307 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.307 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.307 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.307 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.308 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.308 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.308 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.308 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.308 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.308 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.308 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.308 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.309 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.309 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.309 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.309 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.309 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.309 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.309 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.309 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.309 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.310 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.310 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.310 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.310 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.310 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.310 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.310 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.310 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.311 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.311 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.311 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.311 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.311 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.311 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.311 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.311 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.312 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.312 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.312 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.312 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.312 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.312 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.312 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.312 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.313 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.313 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.313 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.313 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.313 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.313 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.313 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.313 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.314 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.314 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.314 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.314 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.314 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.314 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.315 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.315 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.315 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.315 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.315 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.315 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.315 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.316 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.316 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.316 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.316 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.316 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.316 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.316 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.316 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.317 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.317 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.317 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.317 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.317 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.317 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.317 INFO:teuthology.orchestra.run.smithi072.stdout: "organization": { 2024-04-25T13:23:10.317 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "organization", 2024-04-25T13:23:10.318 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.318 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.318 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.318 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.318 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.318 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.318 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.318 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: "proxy": { 2024-04-25T13:23:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "proxy", 2024-04-25T13:23:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.320 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.320 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.320 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.320 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.320 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.320 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.320 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.320 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.321 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.321 INFO:teuthology.orchestra.run.smithi072.stdout: "url": { 2024-04-25T13:23:10.321 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url", 2024-04-25T13:23:10.321 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.321 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.321 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.321 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:23:10.321 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.321 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.322 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.322 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.322 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.322 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.322 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.322 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.322 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.322 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.323 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.323 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "test_orchestrator", 2024-04-25T13:23:10.323 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.323 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.323 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.323 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.323 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.323 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.324 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.324 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.324 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.324 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.324 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.324 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.324 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.325 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.325 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.325 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.325 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.325 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.325 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.325 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.325 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.326 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.326 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.326 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.326 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.326 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.326 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.326 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.327 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.327 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.327 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.327 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.327 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.327 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.327 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.327 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.328 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.328 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.328 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.328 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.328 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.328 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.328 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.328 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.329 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.329 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.329 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.329 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.329 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.329 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.329 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.329 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.329 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.330 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.330 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.330 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.330 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.330 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.330 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.330 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.330 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.331 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.331 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.331 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.331 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.331 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.331 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.331 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.331 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.331 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.332 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.332 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.332 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.332 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.332 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.332 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.332 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-04-25T13:23:10.332 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.333 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.333 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.333 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.333 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.333 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.333 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.333 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.333 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.333 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.334 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.334 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.334 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.334 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.334 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.334 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.334 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.335 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.335 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.335 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.335 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.335 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.335 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.335 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.336 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.336 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.336 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.336 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.336 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.336 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.336 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.336 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.337 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.337 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.337 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.337 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.337 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.337 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.337 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.337 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.337 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.338 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.338 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.338 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.338 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.338 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.338 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.339 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.339 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.339 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.339 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.339 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.339 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.339 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.339 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.340 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.340 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.340 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.340 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.340 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.340 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.340 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.340 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.340 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.341 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.341 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.341 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.341 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.341 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.341 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.341 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.341 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.342 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.342 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_clones": { 2024-04-25T13:23:10.342 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_clones", 2024-04-25T13:23:10.342 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.342 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.342 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.342 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4", 2024-04-25T13:23:10.342 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.343 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.343 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.343 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:23:10.343 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.343 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.343 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.343 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.343 INFO:teuthology.orchestra.run.smithi072.stdout: "periodic_async_work": { 2024-04-25T13:23:10.343 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "periodic_async_work", 2024-04-25T13:23:10.344 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.344 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.344 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.344 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.344 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.344 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.344 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.344 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Periodically check for async work", 2024-04-25T13:23:10.345 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.345 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.345 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.345 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.345 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_delay": { 2024-04-25T13:23:10.345 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:23:10.345 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.346 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.346 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.346 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:10.346 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.346 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.346 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.346 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:23:10.346 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.347 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.347 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.347 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.347 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:23:10.347 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:23:10.347 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.347 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.347 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.348 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.348 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.348 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.348 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.348 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:23:10.348 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.348 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.348 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.349 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.349 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.349 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.349 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.349 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix", 2024-04-25T13:23:10.349 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.349 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.349 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.350 INFO:teuthology.orchestra.run.smithi072.stdout: "discovery_interval": { 2024-04-25T13:23:10.350 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "discovery_interval", 2024-04-25T13:23:10.350 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:10.350 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.350 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.350 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "100", 2024-04-25T13:23:10.350 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.350 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.350 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.351 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.351 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.351 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.351 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.351 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.351 INFO:teuthology.orchestra.run.smithi072.stdout: "identifier": { 2024-04-25T13:23:10.351 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "identifier", 2024-04-25T13:23:10.351 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.352 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.352 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.352 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.352 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.352 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.352 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.352 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.353 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.353 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.353 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.353 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.353 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:10.353 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:10.353 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.353 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.353 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.354 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:10.354 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.354 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.354 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.354 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.354 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.354 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.355 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.355 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.355 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.355 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.355 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.355 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.355 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.355 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.356 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.356 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.356 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.356 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.356 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.356 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.356 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.356 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.356 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.357 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.357 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.357 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.357 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.357 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.357 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.357 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.357 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.358 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.358 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.358 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.358 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.358 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.358 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.358 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.358 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.359 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.359 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.359 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.359 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.359 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.359 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.359 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.359 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.359 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.360 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.360 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.360 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.360 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.360 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.360 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.360 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.360 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.361 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.361 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.361 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.361 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.361 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.361 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.361 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.361 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.362 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.362 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.362 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.362 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.362 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.362 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.362 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.362 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.363 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.363 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.363 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.363 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.363 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.363 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.363 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_host": { 2024-04-25T13:23:10.363 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_host", 2024-04-25T13:23:10.364 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.364 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.364 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.364 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.364 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.364 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.364 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.364 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.365 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.365 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.365 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.365 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.365 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_port": { 2024-04-25T13:23:10.365 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_port", 2024-04-25T13:23:10.365 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.366 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.366 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.366 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10051", 2024-04-25T13:23:10.366 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.366 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.366 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.366 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.366 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.367 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.367 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.367 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.367 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_sender": { 2024-04-25T13:23:10.367 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_sender", 2024-04-25T13:23:10.367 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.367 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.367 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.368 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:23:10.368 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.368 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.368 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.368 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.368 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.368 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.368 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.368 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.369 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.369 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.369 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:10.369 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.369 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.369 INFO:teuthology.orchestra.run.smithi072.stdout: "modules": [ 2024-04-25T13:23:10.369 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-04-25T13:23:10.369 INFO:teuthology.orchestra.run.smithi072.stdout: "dashboard", 2024-04-25T13:23:10.370 INFO:teuthology.orchestra.run.smithi072.stdout: "iostat", 2024-04-25T13:23:10.370 INFO:teuthology.orchestra.run.smithi072.stdout: "nfs", 2024-04-25T13:23:10.370 INFO:teuthology.orchestra.run.smithi072.stdout: "restful", 2024-04-25T13:23:10.370 INFO:teuthology.orchestra.run.smithi072.stdout: "snap_schedule" 2024-04-25T13:23:10.370 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.370 INFO:teuthology.orchestra.run.smithi072.stdout: "available_modules": [ 2024-04-25T13:23:10.370 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.370 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "alerts", 2024-04-25T13:23:10.371 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.371 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.371 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.371 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:10.371 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:10.371 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.371 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.371 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:23:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.373 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.373 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.373 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.373 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.373 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.373 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.373 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.373 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.374 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.374 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.374 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.374 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.374 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.374 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.374 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.374 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.374 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.375 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.375 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.375 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.375 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.375 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.375 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.376 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.376 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.376 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.376 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.376 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.376 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.376 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.376 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.377 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.377 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.377 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.377 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.377 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.377 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.377 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.377 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.378 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.378 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.378 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.378 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.378 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.378 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.378 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.378 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.379 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.379 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.379 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.379 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.379 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.379 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.379 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.379 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.380 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.380 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.380 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.380 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.380 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.380 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.380 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.380 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.381 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.381 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.381 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.381 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.381 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.381 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.381 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.381 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_destination": { 2024-04-25T13:23:10.382 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_destination", 2024-04-25T13:23:10.382 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.382 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.382 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.382 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.382 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.382 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.383 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.383 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:23:10.383 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.383 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.383 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.383 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.383 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_from_name": { 2024-04-25T13:23:10.383 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_from_name", 2024-04-25T13:23:10.384 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.384 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.384 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.384 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "Ceph", 2024-04-25T13:23:10.384 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.384 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.384 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.384 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email From: name", 2024-04-25T13:23:10.385 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.385 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.385 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.385 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.385 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_host": { 2024-04-25T13:23:10.385 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_host", 2024-04-25T13:23:10.385 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.386 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.386 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.386 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.386 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.386 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.386 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.386 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP server", 2024-04-25T13:23:10.386 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.386 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.387 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.387 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.387 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_password": { 2024-04-25T13:23:10.387 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_password", 2024-04-25T13:23:10.387 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.387 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.387 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.388 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.388 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.388 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.388 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.388 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Password to authenticate with", 2024-04-25T13:23:10.388 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.388 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.389 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.389 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.389 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_port": { 2024-04-25T13:23:10.389 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_port", 2024-04-25T13:23:10.389 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.389 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.389 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.389 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "465", 2024-04-25T13:23:10.389 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.390 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.390 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.390 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP port", 2024-04-25T13:23:10.390 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.390 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.390 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.390 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.391 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_sender": { 2024-04-25T13:23:10.391 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_sender", 2024-04-25T13:23:10.391 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.391 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.391 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.391 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.391 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.391 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.392 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.392 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:23:10.392 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.392 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.392 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.392 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.392 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_ssl": { 2024-04-25T13:23:10.392 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_ssl", 2024-04-25T13:23:10.392 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.393 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.393 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.393 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.393 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.393 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.393 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.393 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:23:10.393 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.394 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.394 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.394 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.394 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_user": { 2024-04-25T13:23:10.394 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_user", 2024-04-25T13:23:10.394 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.394 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.395 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.395 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.395 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.395 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.395 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.395 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "User to authenticate as", 2024-04-25T13:23:10.395 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.395 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.396 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.396 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.396 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.396 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.396 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.396 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "balancer", 2024-04-25T13:23:10.396 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.396 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.397 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.397 INFO:teuthology.orchestra.run.smithi072.stdout: "active": { 2024-04-25T13:23:10.397 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "active", 2024-04-25T13:23:10.397 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.397 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.397 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.397 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.397 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.398 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.398 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.398 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:23:10.398 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.398 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.398 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.398 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.398 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_time": { 2024-04-25T13:23:10.398 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_time", 2024-04-25T13:23:10.399 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.399 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.399 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.399 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0000", 2024-04-25T13:23:10.399 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.399 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.399 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.400 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:23:10.400 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:23:10.400 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.400 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.400 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.400 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_weekday": { 2024-04-25T13:23:10.400 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_weekday", 2024-04-25T13:23:10.400 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:10.401 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.401 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.401 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:10.401 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-25T13:23:10.401 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-25T13:23:10.401 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.401 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:23:10.401 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:23:10.402 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.402 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.402 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.402 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_max_iterations": { 2024-04-25T13:23:10.402 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:23:10.402 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:10.402 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.402 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.403 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "25", 2024-04-25T13:23:10.403 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:10.403 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "250", 2024-04-25T13:23:10.403 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.403 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:23:10.403 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.403 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.404 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.404 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.404 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_metrics": { 2024-04-25T13:23:10.404 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_metrics", 2024-04-25T13:23:10.404 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.404 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.404 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.404 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:23:10.404 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.405 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.405 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.405 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:23:10.405 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:23:10.405 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.405 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.405 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.406 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_step": { 2024-04-25T13:23:10.406 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_step", 2024-04-25T13:23:10.406 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.406 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.406 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.406 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.5", 2024-04-25T13:23:10.409 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0.001", 2024-04-25T13:23:10.409 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "0.999", 2024-04-25T13:23:10.409 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.409 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:23:10.409 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:23:10.409 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.410 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.410 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.410 INFO:teuthology.orchestra.run.smithi072.stdout: "end_time": { 2024-04-25T13:23:10.410 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_time", 2024-04-25T13:23:10.410 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.410 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.410 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.410 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2359", 2024-04-25T13:23:10.411 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.411 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.411 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.411 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:23:10.411 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:23:10.411 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.411 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.411 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.411 INFO:teuthology.orchestra.run.smithi072.stdout: "end_weekday": { 2024-04-25T13:23:10.412 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_weekday", 2024-04-25T13:23:10.412 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:10.412 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.412 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.412 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:10.412 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-25T13:23:10.412 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-25T13:23:10.413 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.413 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:23:10.413 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:23:10.413 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.413 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.413 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.413 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.413 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.414 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.414 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.414 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.414 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.414 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.414 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.414 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.414 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.415 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.415 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.415 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.415 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.415 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.415 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.415 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.416 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.416 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.416 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.416 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.416 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.416 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.416 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.416 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.416 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.417 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.417 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.417 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.417 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.417 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.417 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.417 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.418 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.418 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.418 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.418 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.418 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.418 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.418 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.418 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.419 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.419 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.419 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.419 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.419 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.419 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.419 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.419 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.419 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.420 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.420 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.420 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.420 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.420 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.420 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.420 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.420 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.421 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.421 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.421 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.421 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.421 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.421 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.421 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.421 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.422 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.422 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.422 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.422 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.422 INFO:teuthology.orchestra.run.smithi072.stdout: "min_score": { 2024-04-25T13:23:10.422 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_score", 2024-04-25T13:23:10.422 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.422 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.423 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.423 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:10.423 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.423 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.423 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.423 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:23:10.423 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.424 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.424 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.424 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.424 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-25T13:23:10.424 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-25T13:23:10.424 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.424 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.424 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.425 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "upmap", 2024-04-25T13:23:10.425 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.425 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.425 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.425 INFO:teuthology.orchestra.run.smithi072.stdout: "crush-compat", 2024-04-25T13:23:10.425 INFO:teuthology.orchestra.run.smithi072.stdout: "none", 2024-04-25T13:23:10.425 INFO:teuthology.orchestra.run.smithi072.stdout: "read", 2024-04-25T13:23:10.426 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap", 2024-04-25T13:23:10.426 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap-read" 2024-04-25T13:23:10.426 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.426 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Balancer mode", 2024-04-25T13:23:10.426 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.426 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.426 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.426 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.427 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_ids": { 2024-04-25T13:23:10.427 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_ids", 2024-04-25T13:23:10.427 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.427 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.427 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.427 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.427 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.427 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.428 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.428 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:23:10.428 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.428 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.428 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.428 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.428 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:10.428 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:10.428 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.429 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.429 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.429 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:10.429 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.429 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.429 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.429 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:23:10.429 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.430 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.430 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.430 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.430 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_deviation": { 2024-04-25T13:23:10.430 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_deviation", 2024-04-25T13:23:10.430 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.430 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.430 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.431 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:10.431 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:10.431 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.431 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.431 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:23:10.431 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:23:10.431 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.432 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.432 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.432 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_optimizations": { 2024-04-25T13:23:10.432 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:23:10.432 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:10.432 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.432 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.432 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.433 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.433 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.433 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.433 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:23:10.433 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.433 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.433 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.434 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.434 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.434 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.434 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.434 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm", 2024-04-25T13:23:10.434 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.434 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.434 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.435 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_down_multiplier": { 2024-04-25T13:23:10.435 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_down_multiplier", 2024-04-25T13:23:10.435 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.435 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.435 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.435 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-25T13:23:10.435 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.436 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.436 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.436 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:23:10.436 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.436 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.436 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.436 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.437 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_refresh_rate": { 2024-04-25T13:23:10.437 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_refresh_rate", 2024-04-25T13:23:10.437 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.437 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.437 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.437 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "20", 2024-04-25T13:23:10.437 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.437 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.438 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.438 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:23:10.438 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.438 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.438 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.438 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.438 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_starting_port": { 2024-04-25T13:23:10.439 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_starting_port", 2024-04-25T13:23:10.439 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.439 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.439 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.439 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4721", 2024-04-25T13:23:10.439 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.440 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.440 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.440 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:23:10.440 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.440 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.440 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.440 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.440 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_ptrace": { 2024-04-25T13:23:10.441 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_ptrace", 2024-04-25T13:23:10.441 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.441 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.441 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.441 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.441 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.441 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.441 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.442 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:23:10.442 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:23:10.442 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.442 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.442 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.442 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_interval": { 2024-04-25T13:23:10.442 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_interval", 2024-04-25T13:23:10.443 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.443 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.443 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.443 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:10.443 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.443 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.443 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.443 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:23:10.443 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.444 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.444 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.444 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.444 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:23:10.444 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:23:10.444 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.444 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.445 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.445 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.7", 2024-04-25T13:23:10.445 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.445 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.445 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.445 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:23:10.445 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.446 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.446 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.446 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.446 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm_log_destination": { 2024-04-25T13:23:10.446 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm_log_destination", 2024-04-25T13:23:10.446 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.446 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.447 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.447 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.447 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.447 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.447 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.447 INFO:teuthology.orchestra.run.smithi072.stdout: "file", 2024-04-25T13:23:10.447 INFO:teuthology.orchestra.run.smithi072.stdout: "file,syslog", 2024-04-25T13:23:10.448 INFO:teuthology.orchestra.run.smithi072.stdout: "syslog" 2024-04-25T13:23:10.448 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.448 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:23:10.448 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.448 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.448 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.448 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.448 INFO:teuthology.orchestra.run.smithi072.stdout: "cgroups_split": { 2024-04-25T13:23:10.449 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cgroups_split", 2024-04-25T13:23:10.449 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.449 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.449 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.449 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.449 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.449 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.449 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.450 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:23:10.450 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.450 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.450 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.450 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.450 INFO:teuthology.orchestra.run.smithi072.stdout: "config_checks_enabled": { 2024-04-25T13:23:10.450 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_checks_enabled", 2024-04-25T13:23:10.451 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.451 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.451 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.451 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.451 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.451 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.451 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.451 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:23:10.452 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.452 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.452 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.452 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.452 INFO:teuthology.orchestra.run.smithi072.stdout: "config_dashboard": { 2024-04-25T13:23:10.452 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_dashboard", 2024-04-25T13:23:10.452 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.453 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.453 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.453 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.453 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.453 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.453 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.453 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:23:10.454 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.454 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.454 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.454 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.454 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_alertmanager": { 2024-04-25T13:23:10.454 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_alertmanager", 2024-04-25T13:23:10.454 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.455 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.455 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.455 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:23:10.455 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.455 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.455 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.455 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:10.456 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.456 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.456 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.456 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.456 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_base": { 2024-04-25T13:23:10.456 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_base", 2024-04-25T13:23:10.456 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.457 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.457 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.457 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:23:10.457 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.457 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.457 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.457 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:23:10.458 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.458 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.458 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.458 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.458 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_elasticsearch": { 2024-04-25T13:23:10.458 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:23:10.458 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.459 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.459 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.459 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:23:10.459 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.459 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.459 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.459 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "elasticsearch container image", 2024-04-25T13:23:10.459 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.460 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.460 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.460 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.460 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_grafana": { 2024-04-25T13:23:10.460 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_grafana", 2024-04-25T13:23:10.460 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.460 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.460 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.461 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:23:10.461 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.461 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.461 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.461 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:10.461 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.461 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.461 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.462 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.462 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_haproxy": { 2024-04-25T13:23:10.462 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_haproxy", 2024-04-25T13:23:10.462 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.462 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.462 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.462 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:23:10.462 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.463 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.463 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.463 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "HAproxy container image", 2024-04-25T13:23:10.463 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.463 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.463 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.463 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.463 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_agent": { 2024-04-25T13:23:10.464 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:23:10.464 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.464 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.464 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.464 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:23:10.464 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.464 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.465 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.465 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:23:10.465 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.465 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.465 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.465 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.465 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_collector": { 2024-04-25T13:23:10.466 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:23:10.466 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.466 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.466 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.466 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:23:10.466 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.466 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.466 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.467 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:23:10.467 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.467 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.467 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.467 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.467 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_query": { 2024-04-25T13:23:10.467 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:23:10.467 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.468 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.468 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.468 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:23:10.468 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.468 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.468 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.468 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger query container image", 2024-04-25T13:23:10.469 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.469 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.469 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.469 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.469 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_keepalived": { 2024-04-25T13:23:10.469 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_keepalived", 2024-04-25T13:23:10.469 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.469 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.469 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.470 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:23:10.470 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.470 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.470 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.470 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Keepalived container image", 2024-04-25T13:23:10.470 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.470 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.470 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.471 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.471 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_loki": { 2024-04-25T13:23:10.471 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_loki", 2024-04-25T13:23:10.471 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.471 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.471 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.471 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:23:10.471 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.472 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.472 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.472 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Loki container image", 2024-04-25T13:23:10.472 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.472 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.472 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.472 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.472 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_node_exporter": { 2024-04-25T13:23:10.473 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_node_exporter", 2024-04-25T13:23:10.473 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.473 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.473 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.473 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:23:10.473 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.473 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.473 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.473 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:10.474 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.474 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.474 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.474 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.474 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_nvmeof": { 2024-04-25T13:23:10.474 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_nvmeof", 2024-04-25T13:23:10.474 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:23:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Nvme-of container image", 2024-04-25T13:23:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_prometheus": { 2024-04-25T13:23:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_prometheus", 2024-04-25T13:23:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:23:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-25T13:23:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_promtail": { 2024-04-25T13:23:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_promtail", 2024-04-25T13:23:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:23:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Promtail container image", 2024-04-25T13:23:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_samba": { 2024-04-25T13:23:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_samba", 2024-04-25T13:23:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:23:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.481 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.481 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:23:10.481 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.481 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.481 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.481 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.481 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_snmp_gateway": { 2024-04-25T13:23:10.481 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:23:10.481 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.482 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.482 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.482 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:23:10.482 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.482 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.482 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.482 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:23:10.483 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.483 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.483 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.483 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.483 INFO:teuthology.orchestra.run.smithi072.stdout: "container_init": { 2024-04-25T13:23:10.483 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_init", 2024-04-25T13:23:10.483 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.483 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.483 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.484 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.484 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.484 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.484 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.484 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:23:10.484 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.484 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.484 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.485 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.485 INFO:teuthology.orchestra.run.smithi072.stdout: "daemon_cache_timeout": { 2024-04-25T13:23:10.485 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:23:10.485 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.485 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.485 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.485 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:10.485 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.486 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.486 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.486 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:23:10.486 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.486 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.486 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.486 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.486 INFO:teuthology.orchestra.run.smithi072.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:23:10.487 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:23:10.487 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.487 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.487 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.487 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "900", 2024-04-25T13:23:10.487 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.487 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.488 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.488 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:23:10.488 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.488 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.488 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.488 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.488 INFO:teuthology.orchestra.run.smithi072.stdout: "default_registry": { 2024-04-25T13:23:10.488 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_registry", 2024-04-25T13:23:10.488 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.489 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.489 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.489 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io", 2024-04-25T13:23:10.489 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.489 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.489 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.489 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:23:10.489 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.490 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.490 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.490 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.490 INFO:teuthology.orchestra.run.smithi072.stdout: "device_cache_timeout": { 2024-04-25T13:23:10.490 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_cache_timeout", 2024-04-25T13:23:10.490 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.490 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.491 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.491 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1800", 2024-04-25T13:23:10.491 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.491 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.491 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.491 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:23:10.491 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.491 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.492 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.492 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.492 INFO:teuthology.orchestra.run.smithi072.stdout: "device_enhanced_scan": { 2024-04-25T13:23:10.492 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_enhanced_scan", 2024-04-25T13:23:10.492 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.492 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.492 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.492 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.493 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.493 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.493 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.493 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:23:10.493 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.493 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.493 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.494 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.494 INFO:teuthology.orchestra.run.smithi072.stdout: "facts_cache_timeout": { 2024-04-25T13:23:10.494 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "facts_cache_timeout", 2024-04-25T13:23:10.494 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.494 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.494 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.494 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:10.494 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.495 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.495 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.495 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:23:10.495 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.495 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.495 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.495 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.495 INFO:teuthology.orchestra.run.smithi072.stdout: "grafana_dashboards_path": { 2024-04-25T13:23:10.496 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:23:10.496 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.496 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.496 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.496 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:23:10.496 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.496 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.497 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.497 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:23:10.497 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.497 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.497 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.497 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.497 INFO:teuthology.orchestra.run.smithi072.stdout: "host_check_interval": { 2024-04-25T13:23:10.497 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "host_check_interval", 2024-04-25T13:23:10.498 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.498 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.498 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.498 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:10.498 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.498 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.498 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.499 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:23:10.499 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.499 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.499 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.499 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.499 INFO:teuthology.orchestra.run.smithi072.stdout: "hw_monitoring": { 2024-04-25T13:23:10.499 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hw_monitoring", 2024-04-25T13:23:10.499 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.500 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.500 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.500 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.500 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.500 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.500 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.500 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:23:10.500 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.501 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.501 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.501 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.501 INFO:teuthology.orchestra.run.smithi072.stdout: "inventory_list_all": { 2024-04-25T13:23:10.501 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "inventory_list_all", 2024-04-25T13:23:10.501 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.501 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.501 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.502 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.502 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.502 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.502 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.502 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:23:10.502 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.502 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.503 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.503 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.503 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.503 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.503 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.503 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.503 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.503 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.503 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.504 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.504 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.504 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.504 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.504 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.504 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.504 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.504 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.505 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.505 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.505 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.505 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.505 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.505 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.505 INFO:teuthology.orchestra.run.smithi072.stdout: "log_refresh_metadata": { 2024-04-25T13:23:10.505 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_refresh_metadata", 2024-04-25T13:23:10.506 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.506 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.506 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.506 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.506 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.506 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.506 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.507 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:23:10.507 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.507 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.507 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.507 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.507 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.507 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.507 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.508 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.508 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.508 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.508 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.508 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.508 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.508 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:23:10.508 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.509 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.509 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.509 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.509 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.509 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.509 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.509 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.509 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.510 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.510 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.510 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.510 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.510 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.510 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.510 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.510 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.511 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.511 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.511 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.511 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.511 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.511 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.511 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.511 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.512 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.512 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.512 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.512 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.512 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.512 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.512 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.512 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.512 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.513 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.513 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.513 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.513 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.513 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.513 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:23:10.513 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:23:10.513 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.514 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.514 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.514 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.514 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.514 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.514 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.514 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:23:10.514 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.515 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.515 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.515 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.515 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:23:10.515 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:23:10.515 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.515 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.515 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.516 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "*", 2024-04-25T13:23:10.516 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.516 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.516 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.516 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:23:10.516 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.516 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.516 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.517 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.517 INFO:teuthology.orchestra.run.smithi072.stdout: "max_count_per_host": { 2024-04-25T13:23:10.517 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_count_per_host", 2024-04-25T13:23:10.517 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.517 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.517 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.517 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.517 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.518 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.518 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.518 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:23:10.518 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.518 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.518 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.518 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.518 INFO:teuthology.orchestra.run.smithi072.stdout: "max_osd_draining_count": { 2024-04-25T13:23:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_osd_draining_count", 2024-04-25T13:23:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:23:10.520 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.520 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.520 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.520 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.520 INFO:teuthology.orchestra.run.smithi072.stdout: "migration_current": { 2024-04-25T13:23:10.520 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "migration_current", 2024-04-25T13:23:10.520 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.521 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.521 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.521 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.521 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.521 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.521 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.521 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "internal - do not modify", 2024-04-25T13:23:10.522 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.522 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.522 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.522 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.522 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-25T13:23:10.522 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-25T13:23:10.522 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.522 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.523 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.523 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "root", 2024-04-25T13:23:10.523 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.523 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.523 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.523 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm-package", 2024-04-25T13:23:10.523 INFO:teuthology.orchestra.run.smithi072.stdout: "root" 2024-04-25T13:23:10.524 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.524 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:23:10.524 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.524 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.524 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.524 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.524 INFO:teuthology.orchestra.run.smithi072.stdout: "oob_default_addr": { 2024-04-25T13:23:10.524 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "oob_default_addr", 2024-04-25T13:23:10.525 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.525 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.525 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.525 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "169.254.1.1", 2024-04-25T13:23:10.525 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.525 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.525 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.526 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:23:10.526 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.526 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.526 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.526 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.526 INFO:teuthology.orchestra.run.smithi072.stdout: "prometheus_alerts_path": { 2024-04-25T13:23:10.526 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:23:10.527 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.527 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.527 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.527 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:23:10.527 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.527 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.527 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.528 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:23:10.528 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.528 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.528 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.528 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.528 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_insecure": { 2024-04-25T13:23:10.528 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_insecure", 2024-04-25T13:23:10.528 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.529 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.529 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.529 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.529 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.529 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.529 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.529 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:23:10.529 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.530 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.530 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.530 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.530 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_password": { 2024-04-25T13:23:10.530 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_password", 2024-04-25T13:23:10.530 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.530 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.531 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.531 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.531 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.531 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.531 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.531 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:23:10.531 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.532 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.532 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.532 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.532 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_url": { 2024-04-25T13:23:10.532 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_url", 2024-04-25T13:23:10.532 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.532 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.532 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.533 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.533 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.533 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.533 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.533 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:23:10.533 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.533 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.533 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.534 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.534 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_username": { 2024-04-25T13:23:10.534 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_username", 2024-04-25T13:23:10.534 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.534 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.534 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.534 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.535 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.535 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.535 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.535 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:23:10.535 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.535 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.535 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.535 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.536 INFO:teuthology.orchestra.run.smithi072.stdout: "secure_monitoring_stack": { 2024-04-25T13:23:10.536 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:23:10.536 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.536 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.536 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.536 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.536 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.537 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.537 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.537 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:23:10.537 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.537 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.537 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.537 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.538 INFO:teuthology.orchestra.run.smithi072.stdout: "service_discovery_port": { 2024-04-25T13:23:10.538 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "service_discovery_port", 2024-04-25T13:23:10.538 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.538 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.538 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.538 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8765", 2024-04-25T13:23:10.538 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.538 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.539 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.539 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:23:10.539 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.539 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.539 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.539 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.539 INFO:teuthology.orchestra.run.smithi072.stdout: "ssh_config_file": { 2024-04-25T13:23:10.539 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssh_config_file", 2024-04-25T13:23:10.540 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.540 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.540 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.540 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.540 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.540 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.540 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.540 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:23:10.541 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.541 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.541 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.541 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.541 INFO:teuthology.orchestra.run.smithi072.stdout: "use_agent": { 2024-04-25T13:23:10.541 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_agent", 2024-04-25T13:23:10.541 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.541 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.542 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.542 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.542 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.542 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.542 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.542 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:23:10.542 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.542 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.542 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.543 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.543 INFO:teuthology.orchestra.run.smithi072.stdout: "use_repo_digest": { 2024-04-25T13:23:10.543 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_repo_digest", 2024-04-25T13:23:10.543 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.543 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.543 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.543 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.543 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:23:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_failed_host_check": { 2024-04-25T13:23:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:23:10.545 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.545 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.545 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.545 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.545 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.545 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.545 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.545 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:23:10.546 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.546 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.546 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.546 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.546 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_daemons": { 2024-04-25T13:23:10.546 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:23:10.546 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.547 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.547 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.547 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.547 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.547 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.547 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.547 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:23:10.547 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.547 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.548 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.548 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.548 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_hosts": { 2024-04-25T13:23:10.548 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:23:10.548 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.548 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.548 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.548 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.549 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.549 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.549 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.549 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:23:10.549 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.549 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.549 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.549 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.550 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.550 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.550 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.550 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crash", 2024-04-25T13:23:10.550 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.550 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.550 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.550 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.551 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.551 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.551 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.551 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.551 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.551 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.551 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.551 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.552 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.552 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.552 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.552 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.552 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.552 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.552 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.552 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.553 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.553 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.553 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.553 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.553 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.553 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.553 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.553 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.554 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.554 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.554 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.554 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.554 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.554 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.554 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.555 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.555 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.555 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.555 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.555 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.555 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.555 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.556 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.556 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.556 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.556 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.556 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.556 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.556 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.557 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.557 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.557 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.557 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.557 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.557 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.557 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.557 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.558 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.558 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.558 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.558 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.558 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.558 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.558 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.558 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.559 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.559 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.559 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.559 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.559 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.559 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.559 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.559 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.560 INFO:teuthology.orchestra.run.smithi072.stdout: "retain_interval": { 2024-04-25T13:23:10.560 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retain_interval", 2024-04-25T13:23:10.560 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.560 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.560 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.560 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "31536000", 2024-04-25T13:23:10.560 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.560 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.561 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.561 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:23:10.561 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.561 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.561 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.561 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.561 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_recent_interval": { 2024-04-25T13:23:10.562 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_recent_interval", 2024-04-25T13:23:10.562 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.562 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.562 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.562 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1209600", 2024-04-25T13:23:10.562 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.562 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.563 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.563 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:23:10.563 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.563 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.563 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.563 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.563 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.564 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.564 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.564 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dashboard", 2024-04-25T13:23:10.564 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.564 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.564 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.564 INFO:teuthology.orchestra.run.smithi072.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:23:10.564 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:23:10.565 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.565 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.565 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.565 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.565 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.565 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.565 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.566 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.566 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.566 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.566 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.566 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.566 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:23:10.566 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:23:10.567 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.567 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.567 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.567 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.567 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.567 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.567 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.567 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.568 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.568 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.568 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.568 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.568 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:23:10.568 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:23:10.568 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.568 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.569 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.569 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.569 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.569 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.569 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.569 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.569 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.569 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.570 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.570 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.570 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:23:10.570 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:23:10.570 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.570 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.570 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.571 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.571 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.571 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.571 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.571 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.571 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.571 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.571 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.572 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.572 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:23:10.572 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:23:10.572 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.572 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.572 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.572 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.572 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.573 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.573 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.573 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.573 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.573 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.573 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.573 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.573 INFO:teuthology.orchestra.run.smithi072.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:23:10.574 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:23:10.574 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.574 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.574 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.574 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.574 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.574 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.575 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.575 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.575 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.575 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.575 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.575 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.575 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:23:10.576 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:23:10.576 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.576 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.576 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.576 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.576 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.576 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.576 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.577 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.577 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.577 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.577 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.577 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.577 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:23:10.577 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:23:10.577 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.578 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.578 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.578 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.578 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.578 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.578 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.578 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.579 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.579 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.579 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.579 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.579 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:23:10.579 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:23:10.579 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.580 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.580 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.580 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.580 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.580 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.580 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.580 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.580 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.581 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.581 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.581 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.581 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:23:10.581 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:23:10.581 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.581 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.581 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.582 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.582 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.582 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.582 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.582 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.582 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.582 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.583 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.583 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.583 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:23:10.583 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:23:10.583 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.583 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.583 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.583 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.585 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.585 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:23:10.585 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:23:10.585 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.585 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.585 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.585 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.585 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.586 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.586 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.586 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.586 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.586 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.586 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.586 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.586 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:23:10.587 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:23:10.587 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.587 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.587 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.587 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.587 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.587 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.588 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.588 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.588 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.588 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.588 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.588 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.588 INFO:teuthology.orchestra.run.smithi072.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:23:10.588 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:23:10.589 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.589 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.589 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.589 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.589 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.589 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.589 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.589 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.590 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.590 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.590 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.590 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.590 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:23:10.590 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:23:10.590 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.591 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.591 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.591 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-25T13:23:10.591 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.591 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.591 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.591 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.592 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.592 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.592 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.592 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.592 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:23:10.592 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:23:10.592 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.592 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.593 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.593 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.593 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.593 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.593 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.593 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.593 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.593 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.594 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.594 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.594 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_URL": { 2024-04-25T13:23:10.594 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:23:10.594 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.594 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.594 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.594 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.595 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.595 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.595 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.595 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.595 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.595 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.595 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.595 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.596 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:23:10.596 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:23:10.596 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.596 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.596 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.596 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-25T13:23:10.596 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.597 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.597 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.597 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.597 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.597 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.597 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.597 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.597 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:23:10.598 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:23:10.598 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.598 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.598 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.598 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.598 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.598 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.598 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.599 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.599 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.599 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.599 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.599 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.599 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:23:10.599 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:23:10.600 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.600 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.600 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.600 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.600 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.600 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.600 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.601 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.601 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.601 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.601 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.601 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.601 INFO:teuthology.orchestra.run.smithi072.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:23:10.601 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:23:10.601 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.602 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.602 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.602 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.602 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.602 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.602 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.603 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.603 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.603 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.603 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.603 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.603 INFO:teuthology.orchestra.run.smithi072.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:23:10.603 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:23:10.603 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.604 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.604 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.604 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.604 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.604 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.604 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.604 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.605 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.605 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.605 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.605 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.605 INFO:teuthology.orchestra.run.smithi072.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:23:10.605 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:23:10.605 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.605 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.606 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.606 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "[]", 2024-04-25T13:23:10.606 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.606 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.606 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.606 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.606 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.607 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.607 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.607 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.607 INFO:teuthology.orchestra.run.smithi072.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:23:10.607 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:23:10.607 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.607 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.607 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.608 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "{}", 2024-04-25T13:23:10.608 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.608 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.608 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.608 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.608 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.608 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.608 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.608 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.609 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:23:10.609 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:23:10.609 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.609 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.609 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.609 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.609 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.609 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.610 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.610 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.610 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.610 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.610 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.610 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.610 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:23:10.610 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:23:10.611 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.611 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.611 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.611 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.613 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.613 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.613 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.613 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.613 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.614 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.614 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.614 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.614 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:23:10.614 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:23:10.614 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.614 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.615 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.615 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.615 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.615 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.615 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.615 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.615 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.616 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.616 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.616 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.616 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:23:10.616 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:23:10.616 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.616 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.616 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.616 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.617 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.617 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.617 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.617 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.617 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.617 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.617 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.617 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.617 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:23:10.618 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:23:10.618 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.618 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.618 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.618 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.618 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.618 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.618 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.619 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.619 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.619 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.619 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.619 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.619 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:23:10.619 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:23:10.619 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.620 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.620 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.620 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.620 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.620 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.620 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.620 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.620 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.621 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.621 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.621 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.621 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:23:10.621 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:23:10.621 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.621 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.621 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.622 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.622 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.622 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.622 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.622 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.622 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.622 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.622 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.623 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.623 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:23:10.623 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:23:10.623 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.623 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.623 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.623 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.623 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.623 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.624 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.624 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.624 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.624 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.624 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.624 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.624 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:23:10.625 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:23:10.625 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.625 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.625 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.625 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.625 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.625 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.625 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.626 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.626 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.626 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.626 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.626 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.626 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:23:10.626 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:23:10.626 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.627 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.627 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.627 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.627 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.627 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.627 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.627 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.627 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.628 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.628 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.628 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.628 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:23:10.628 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:23:10.628 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.628 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.628 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.628 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:23:10.629 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.629 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.629 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.629 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.629 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.629 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.629 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.629 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.630 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:23:10.630 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:23:10.630 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.630 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.630 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.630 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.630 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.630 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.630 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.631 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.631 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.631 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.631 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.631 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.631 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:23:10.631 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:23:10.631 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.632 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.632 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.632 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8", 2024-04-25T13:23:10.632 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.632 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.632 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.632 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.632 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.633 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.633 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.633 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.633 INFO:teuthology.orchestra.run.smithi072.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:23:10.633 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:23:10.633 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.633 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.633 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.633 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "45", 2024-04-25T13:23:10.634 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.634 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.634 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.634 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.634 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.634 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.634 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.635 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.635 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:23:10.635 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:23:10.635 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.635 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.635 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.635 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.635 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.635 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.636 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.636 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.636 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.636 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.636 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.636 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.636 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:23:10.636 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:23:10.637 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.637 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.637 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.637 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-25T13:23:10.637 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.637 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.637 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.637 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.638 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.638 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.638 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.638 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.638 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:23:10.638 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:23:10.638 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.638 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.639 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.639 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.639 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.639 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.639 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.639 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.639 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.639 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.639 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.640 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.640 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:23:10.640 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:23:10.640 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.640 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.640 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.640 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.640 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.641 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.641 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.641 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.641 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.641 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.641 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.641 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.641 INFO:teuthology.orchestra.run.smithi072.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:23:10.642 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:23:10.642 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.642 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.642 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.642 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.642 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.642 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.642 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.643 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.643 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.643 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.643 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.643 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.643 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:23:10.643 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:23:10.643 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.644 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.644 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.644 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:10.644 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.644 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.644 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.644 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.645 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.645 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.645 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.645 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.645 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:23:10.645 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:23:10.645 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.645 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.646 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.646 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.646 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.646 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.646 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.646 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.646 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.647 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.647 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.647 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.647 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:23:10.647 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:23:10.647 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.647 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.647 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.648 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:10.648 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.648 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.648 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.648 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.648 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.648 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.649 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.649 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.649 INFO:teuthology.orchestra.run.smithi072.stdout: "cross_origin_url": { 2024-04-25T13:23:10.649 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cross_origin_url", 2024-04-25T13:23:10.649 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.649 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.649 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.649 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.650 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.650 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.650 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.650 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.650 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.650 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.650 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.651 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.651 INFO:teuthology.orchestra.run.smithi072.stdout: "crt_file": { 2024-04-25T13:23:10.651 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crt_file", 2024-04-25T13:23:10.651 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.651 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.651 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.651 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.651 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.652 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.652 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.652 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.652 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.652 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.652 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.652 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.652 INFO:teuthology.orchestra.run.smithi072.stdout: "debug": { 2024-04-25T13:23:10.652 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "debug", 2024-04-25T13:23:10.653 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.653 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.653 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.653 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.653 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.653 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.653 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.653 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:23:10.653 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.654 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.654 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.654 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.654 INFO:teuthology.orchestra.run.smithi072.stdout: "jwt_token_ttl": { 2024-04-25T13:23:10.654 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "jwt_token_ttl", 2024-04-25T13:23:10.654 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.654 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "28800", 2024-04-25T13:23:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.656 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.656 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.656 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-25T13:23:10.656 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-25T13:23:10.656 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.656 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.656 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.656 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.657 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.657 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.657 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.657 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.657 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.657 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.657 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.657 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.657 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.658 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.658 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.658 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.658 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.658 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.658 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.658 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.658 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.659 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.659 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.659 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.659 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.659 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.659 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.659 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.659 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.660 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.660 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.660 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.660 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.660 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.660 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.660 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.660 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.660 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.661 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.661 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.661 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.661 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.661 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.661 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.661 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.661 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.662 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.662 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.662 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.662 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.662 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.662 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.662 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.662 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.663 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.663 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.663 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.663 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.663 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.663 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.663 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.663 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.664 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.664 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.664 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.664 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.664 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.664 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.664 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.664 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.665 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.665 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.665 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.665 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.665 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.665 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.665 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.666 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.666 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.666 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.666 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.666 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.666 INFO:teuthology.orchestra.run.smithi072.stdout: "motd": { 2024-04-25T13:23:10.666 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "motd", 2024-04-25T13:23:10.666 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.667 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.667 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.667 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.667 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.667 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.667 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.667 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "The message of the day", 2024-04-25T13:23:10.667 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.668 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.668 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.668 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.668 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:23:10.668 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:23:10.668 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.668 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.669 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.669 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.669 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.669 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.669 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.669 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.669 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.669 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.670 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.670 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.670 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-25T13:23:10.670 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-25T13:23:10.670 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.670 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.670 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.670 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-25T13:23:10.671 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.671 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.671 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.671 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.671 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.671 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.671 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.671 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.672 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-25T13:23:10.672 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-25T13:23:10.672 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.672 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.672 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.672 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8080", 2024-04-25T13:23:10.672 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.672 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.672 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.673 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.673 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.673 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.673 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.673 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.673 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-25T13:23:10.673 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-25T13:23:10.673 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.674 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.674 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.674 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.674 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.674 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.674 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.674 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.674 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.675 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.675 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.675 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.675 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl_server_port": { 2024-04-25T13:23:10.675 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl_server_port", 2024-04-25T13:23:10.675 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.675 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.675 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.676 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8443", 2024-04-25T13:23:10.676 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.676 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.676 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.676 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.676 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.676 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.676 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.677 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.677 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-25T13:23:10.677 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-25T13:23:10.677 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.677 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.677 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.677 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "redirect", 2024-04-25T13:23:10.677 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.678 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.678 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.678 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.678 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect" 2024-04-25T13:23:10.678 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.678 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.678 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.679 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.679 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.679 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.679 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-25T13:23:10.679 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-25T13:23:10.679 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.679 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.679 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.680 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-25T13:23:10.680 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-25T13:23:10.680 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-25T13:23:10.680 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.680 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.680 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.680 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.680 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.681 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.681 INFO:teuthology.orchestra.run.smithi072.stdout: "url_prefix": { 2024-04-25T13:23:10.681 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url_prefix", 2024-04-25T13:23:10.681 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.681 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.681 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.681 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.681 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.681 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.682 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.682 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.682 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.682 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.682 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.682 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.682 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.682 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.683 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.683 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-04-25T13:23:10.683 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.683 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.683 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.683 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_monitoring": { 2024-04-25T13:23:10.683 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_monitoring", 2024-04-25T13:23:10.683 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.684 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.684 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.684 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.684 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.684 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.684 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.684 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "monitor device health metrics", 2024-04-25T13:23:10.684 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.685 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.685 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.685 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.685 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.685 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.685 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.685 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.686 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.686 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.686 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.686 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.686 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.686 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.686 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.687 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.687 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.687 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.687 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.687 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.687 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.687 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.688 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.688 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.688 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.688 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.688 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.688 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.688 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.688 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.689 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.689 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.689 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.689 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.689 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.689 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.689 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.689 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.690 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.690 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.690 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.690 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.690 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.690 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.690 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.690 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.691 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.691 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.691 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.691 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.691 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.691 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.691 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.691 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.692 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.692 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.692 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.692 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.692 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.692 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.692 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.693 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.693 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.693 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.693 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.693 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.693 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.693 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.693 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.694 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.694 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.694 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.694 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.694 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.694 INFO:teuthology.orchestra.run.smithi072.stdout: "mark_out_threshold": { 2024-04-25T13:23:10.694 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mark_out_threshold", 2024-04-25T13:23:10.694 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.695 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.695 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.695 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2419200", 2024-04-25T13:23:10.695 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.695 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.695 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.695 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:23:10.696 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.696 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.696 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.696 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.696 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_name": { 2024-04-25T13:23:10.696 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_name", 2024-04-25T13:23:10.696 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.696 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.696 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.697 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "device_health_metrics", 2024-04-25T13:23:10.697 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.697 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.697 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.697 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:23:10.697 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.697 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.697 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.698 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.698 INFO:teuthology.orchestra.run.smithi072.stdout: "retention_period": { 2024-04-25T13:23:10.698 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retention_period", 2024-04-25T13:23:10.698 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.698 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.698 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.698 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15552000", 2024-04-25T13:23:10.698 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.699 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.699 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.699 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:23:10.699 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.699 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.699 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.700 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.700 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_frequency": { 2024-04-25T13:23:10.700 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_frequency", 2024-04-25T13:23:10.700 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.700 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.700 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.700 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-25T13:23:10.700 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.701 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.701 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.701 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:23:10.701 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.701 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.701 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.701 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.701 INFO:teuthology.orchestra.run.smithi072.stdout: "self_heal": { 2024-04-25T13:23:10.701 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "self_heal", 2024-04-25T13:23:10.702 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.702 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.702 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.702 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.702 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.702 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.702 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.702 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:23:10.703 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.703 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.703 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.703 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.703 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:10.703 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:10.703 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.703 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:23:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.705 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.705 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.705 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_threshold": { 2024-04-25T13:23:10.705 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_threshold", 2024-04-25T13:23:10.705 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.705 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.705 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.705 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7257600", 2024-04-25T13:23:10.706 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.706 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.706 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.706 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:23:10.706 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.706 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.706 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.706 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.706 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.707 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.707 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.707 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "diskprediction_local", 2024-04-25T13:23:10.707 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.707 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.707 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.707 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.707 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.708 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.708 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.708 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.708 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.708 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.708 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.708 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.708 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.709 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.709 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.709 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.709 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.709 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.709 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.709 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.709 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.709 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.710 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.710 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.710 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.710 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.710 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.710 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.710 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.710 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.711 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.711 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.711 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.711 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.711 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.711 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.711 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.711 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.712 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.712 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.712 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.712 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.712 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.712 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.712 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.712 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.712 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.713 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.713 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.713 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.713 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.713 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.713 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.713 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.713 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.714 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.714 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.714 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.714 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.714 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.714 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.714 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.716 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.716 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.716 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.716 INFO:teuthology.orchestra.run.smithi072.stdout: "predict_interval": { 2024-04-25T13:23:10.716 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predict_interval", 2024-04-25T13:23:10.716 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.716 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.716 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-25T13:23:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.718 INFO:teuthology.orchestra.run.smithi072.stdout: "predictor_model": { 2024-04-25T13:23:10.718 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predictor_model", 2024-04-25T13:23:10.718 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.718 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.718 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.718 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "prophetstor", 2024-04-25T13:23:10.718 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.718 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.719 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.719 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.719 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.719 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.719 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.719 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.719 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:10.720 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:10.720 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.720 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.720 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.720 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-25T13:23:10.720 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.720 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.720 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.720 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.721 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.721 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.721 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.721 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.721 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.721 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.721 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.721 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "influx", 2024-04-25T13:23:10.722 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": false, 2024-04-25T13:23:10.722 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:23:10.722 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.722 INFO:teuthology.orchestra.run.smithi072.stdout: "batch_size": { 2024-04-25T13:23:10.722 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "batch_size", 2024-04-25T13:23:10.722 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.722 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.722 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.723 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5000", 2024-04-25T13:23:10.723 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.723 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.723 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.723 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:23:10.723 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.723 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.724 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.724 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.724 INFO:teuthology.orchestra.run.smithi072.stdout: "database": { 2024-04-25T13:23:10.724 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "database", 2024-04-25T13:23:10.724 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.724 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.724 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.725 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "ceph", 2024-04-25T13:23:10.725 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.725 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.725 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.725 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:23:10.725 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.725 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.726 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.726 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.726 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": { 2024-04-25T13:23:10.726 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hostname", 2024-04-25T13:23:10.726 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.726 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.726 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.726 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.726 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.727 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.727 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.727 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:23:10.727 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.727 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.727 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.727 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.727 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:10.728 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:10.728 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.728 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.728 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.728 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "30", 2024-04-25T13:23:10.728 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "5", 2024-04-25T13:23:10.728 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.728 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.729 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:23:10.729 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.729 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.729 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.729 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.729 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.729 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.730 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.730 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.730 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.730 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.730 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.730 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.730 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.730 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.731 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.731 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.731 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.731 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.731 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.731 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.731 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.731 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.732 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.732 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.732 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.732 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.732 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.732 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.732 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.732 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.732 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.733 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.733 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.733 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.733 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.733 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.733 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.733 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.734 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.734 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.734 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.734 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.734 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.734 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.734 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.734 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.734 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.735 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.735 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.735 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.735 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.735 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.735 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.735 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.736 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.736 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.736 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.736 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.736 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.736 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.736 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.736 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.736 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.737 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.737 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.737 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.737 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.737 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.737 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.737 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.738 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.738 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.738 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.738 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.738 INFO:teuthology.orchestra.run.smithi072.stdout: "password": { 2024-04-25T13:23:10.738 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "password", 2024-04-25T13:23:10.738 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.738 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.739 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.739 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.739 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.739 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.739 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.739 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:23:10.739 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.739 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.740 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.740 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.740 INFO:teuthology.orchestra.run.smithi072.stdout: "port": { 2024-04-25T13:23:10.740 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "port", 2024-04-25T13:23:10.740 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.740 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.740 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.740 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8086", 2024-04-25T13:23:10.741 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.741 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.741 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.741 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server port", 2024-04-25T13:23:10.741 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.741 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.741 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.742 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.742 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-25T13:23:10.742 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-25T13:23:10.742 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.742 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.742 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.742 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "false", 2024-04-25T13:23:10.743 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.743 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.743 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.743 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:23:10.743 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.743 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.743 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.744 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.744 INFO:teuthology.orchestra.run.smithi072.stdout: "threads": { 2024-04-25T13:23:10.744 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threads", 2024-04-25T13:23:10.744 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.744 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.744 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.744 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:10.744 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:10.745 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "32", 2024-04-25T13:23:10.745 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.745 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:23:10.745 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.745 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.745 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.745 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.745 INFO:teuthology.orchestra.run.smithi072.stdout: "username": { 2024-04-25T13:23:10.746 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "username", 2024-04-25T13:23:10.746 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.746 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.746 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.746 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.746 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.746 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.747 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.747 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:23:10.747 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.747 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.747 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.747 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.747 INFO:teuthology.orchestra.run.smithi072.stdout: "verify_ssl": { 2024-04-25T13:23:10.747 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "verify_ssl", 2024-04-25T13:23:10.748 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.748 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.748 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.748 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "true", 2024-04-25T13:23:10.748 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.748 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.748 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.749 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:23:10.749 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.749 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.749 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.749 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.749 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.749 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.749 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.750 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "insights", 2024-04-25T13:23:10.750 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.750 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.750 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.750 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.750 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.751 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.751 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.751 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.751 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.751 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.751 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.751 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.752 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.752 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.752 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.752 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.752 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.752 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.752 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.752 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.753 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.753 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.753 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.753 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.753 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.753 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.753 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.753 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.754 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.754 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.754 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.754 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.754 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.754 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.754 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.755 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.755 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.755 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.755 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.755 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.755 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.755 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.756 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.756 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.756 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.756 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.756 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.756 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.756 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.756 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.757 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.757 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.757 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.757 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.757 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.757 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.757 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.757 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.758 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.758 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.758 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.758 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.758 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.758 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.758 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.759 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.759 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.759 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.759 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.759 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.759 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.759 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.759 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.760 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.760 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.760 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.760 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "iostat", 2024-04-25T13:23:10.760 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.760 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.760 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.760 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.761 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.761 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.761 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.761 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.761 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.761 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.761 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.761 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.761 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.762 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.762 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.762 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.762 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.762 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.762 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.762 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.763 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.763 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.763 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.763 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.763 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.763 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.763 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.763 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.763 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.764 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.764 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.764 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.764 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.764 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.764 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.764 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.764 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.765 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.765 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.765 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.765 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.765 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.765 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.765 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.766 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.766 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.766 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.766 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.766 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.766 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.766 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.767 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.767 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.767 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.767 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.767 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.767 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.767 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.767 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.768 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.768 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.768 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.768 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.768 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.768 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.768 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.768 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.769 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.769 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.769 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.769 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.769 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.769 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.769 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.769 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.770 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.770 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "k8sevents", 2024-04-25T13:23:10.770 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.770 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.770 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.770 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_event_retention_days": { 2024-04-25T13:23:10.770 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:23:10.771 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.771 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.771 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.771 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7", 2024-04-25T13:23:10.771 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.771 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.771 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.771 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:23:10.771 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.772 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.772 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.772 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.772 INFO:teuthology.orchestra.run.smithi072.stdout: "config_check_secs": { 2024-04-25T13:23:10.772 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_check_secs", 2024-04-25T13:23:10.772 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.772 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.772 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.773 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.773 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "10", 2024-04-25T13:23:10.773 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.773 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.773 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:23:10.773 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.773 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.774 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.774 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.774 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.774 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.774 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.774 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.774 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.775 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.775 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.775 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.775 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.775 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.775 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.775 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.775 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.776 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.776 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.776 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.776 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.776 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.776 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.776 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.776 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.777 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.777 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.777 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.777 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.777 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.777 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.777 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.777 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.777 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.778 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.778 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.778 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.778 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.778 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.778 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.778 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.779 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.779 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.779 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.779 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.779 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.779 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.779 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.779 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.780 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.780 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.780 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.780 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.780 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.780 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.780 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.780 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.781 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.781 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.781 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.781 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.781 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.781 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.781 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.781 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.781 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.782 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.782 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.782 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.782 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.782 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.782 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.782 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.783 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.783 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.783 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.783 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.783 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "localpool", 2024-04-25T13:23:10.783 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.785 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.785 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.786 INFO:teuthology.orchestra.run.smithi072.stdout: "failure_domain": { 2024-04-25T13:23:10.786 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "failure_domain", 2024-04-25T13:23:10.786 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.786 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.786 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.786 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "host", 2024-04-25T13:23:10.786 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.786 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.787 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.787 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:23:10.787 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:23:10.787 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.787 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.787 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.787 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.787 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.788 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.788 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.788 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.788 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.788 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.788 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.788 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.788 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.788 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.789 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.789 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.789 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.789 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.789 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.789 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.789 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.789 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.789 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.790 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.790 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.790 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.790 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.790 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.790 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.790 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.790 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.791 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.791 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.791 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.791 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.791 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.791 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.791 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.791 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.791 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.792 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.793 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.793 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.793 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.793 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.793 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.793 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.793 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.793 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.793 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.794 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.794 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.794 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.794 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.794 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.794 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.794 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.794 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.794 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.795 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.795 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.795 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.795 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.795 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.795 INFO:teuthology.orchestra.run.smithi072.stdout: "min_size": { 2024-04-25T13:23:10.795 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_size", 2024-04-25T13:23:10.795 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.795 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.796 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.797 INFO:teuthology.orchestra.run.smithi072.stdout: "num_rep": { 2024-04-25T13:23:10.797 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "num_rep", 2024-04-25T13:23:10.797 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.797 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.797 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.797 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3", 2024-04-25T13:23:10.797 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.797 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.797 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.798 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:23:10.798 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.798 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.798 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.798 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.798 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_num": { 2024-04-25T13:23:10.798 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_num", 2024-04-25T13:23:10.798 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.798 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.799 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.799 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "128", 2024-04-25T13:23:10.799 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.799 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.799 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.799 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:23:10.799 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.799 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.799 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: "prefix": { 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prefix", 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.800 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.801 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:23:10.801 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.801 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.801 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.801 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.801 INFO:teuthology.orchestra.run.smithi072.stdout: "subtree": { 2024-04-25T13:23:10.801 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "subtree", 2024-04-25T13:23:10.801 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.801 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.802 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.802 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "rack", 2024-04-25T13:23:10.802 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.802 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.802 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.802 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:23:10.802 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:23:10.802 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.802 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mds_autoscaler", 2024-04-25T13:23:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.804 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.804 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.804 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.804 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.804 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.804 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.804 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.804 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.804 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.805 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.805 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.805 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.805 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.805 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.805 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.805 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.805 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.805 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.806 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.806 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.806 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.806 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.806 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.806 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.806 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.806 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.806 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.807 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.808 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.808 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.808 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.808 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.808 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.808 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.808 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.808 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.808 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.809 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.810 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.810 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.810 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.810 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.810 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.810 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.810 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.810 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirroring", 2024-04-25T13:23:10.811 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.812 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.813 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.813 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.813 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.813 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.813 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.813 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.813 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.813 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.813 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.814 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.814 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.814 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.814 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.814 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.814 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.814 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.814 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.815 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.815 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.815 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.815 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.815 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.815 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.815 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.815 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.815 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.816 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.816 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.816 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.816 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.816 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.816 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.816 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.816 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.816 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.817 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.819 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.819 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.819 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.819 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.819 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.819 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.819 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.819 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.819 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "nfs", 2024-04-25T13:23:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.821 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.821 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.821 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.821 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.821 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.821 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.821 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.821 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.821 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.822 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.822 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.822 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.822 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.822 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.822 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.822 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.822 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.822 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.823 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.823 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.823 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.823 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.823 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.823 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.823 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.823 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.823 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.824 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.824 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.824 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.824 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.824 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.824 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.824 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.824 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.824 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.825 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.825 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.825 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.825 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.825 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.825 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.825 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.825 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.825 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.826 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.826 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.826 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.826 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.826 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.826 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.826 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.826 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.826 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.827 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.828 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.828 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.828 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.828 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.828 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.828 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.828 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-25T13:23:10.828 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.828 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.829 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.829 INFO:teuthology.orchestra.run.smithi072.stdout: "fail_fs": { 2024-04-25T13:23:10.829 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "fail_fs", 2024-04-25T13:23:10.829 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.829 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.829 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.829 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.829 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.829 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.830 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.830 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:23:10.830 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.830 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.830 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.830 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.830 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.830 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.830 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.831 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.832 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.832 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.832 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.832 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.832 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.832 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.832 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.832 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.832 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.833 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.834 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.834 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.834 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.834 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.834 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.834 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.834 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.834 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.834 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.835 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.835 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.835 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.835 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.835 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.835 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.835 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.835 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.835 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.836 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.836 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.836 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.836 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.836 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.836 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.836 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.836 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.836 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.837 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.838 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.838 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator": { 2024-04-25T13:23:10.838 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-25T13:23:10.838 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.838 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.838 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.838 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.838 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.838 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: "rook", 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: "test_orchestrator" 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Orchestrator backend", 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.839 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.840 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.840 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.840 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.840 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_perf_query", 2024-04-25T13:23:10.840 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.840 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.840 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.840 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.841 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.841 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.841 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.841 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.841 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.841 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.841 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.841 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.841 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.842 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.842 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.842 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.842 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.842 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.842 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.842 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.842 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.842 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.843 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.843 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.843 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.843 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.843 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.843 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.843 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.843 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.843 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.844 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.845 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.845 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.845 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.845 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.845 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.845 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.845 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.845 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.845 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.846 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.846 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.846 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.846 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.846 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.846 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.846 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.846 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.847 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.848 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.848 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.848 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.848 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.848 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.848 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.848 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.848 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.848 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_support", 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.849 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.850 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.850 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.850 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.850 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.850 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.850 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.850 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.850 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.850 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.851 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.851 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.851 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.851 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.851 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.851 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.851 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.851 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.851 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.852 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.853 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.853 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.853 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.853 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.853 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.853 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.853 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.853 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.853 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.854 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.854 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.854 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.854 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.854 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.854 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.854 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.854 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.854 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.855 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.855 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.855 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.855 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.855 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.855 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.855 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.855 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.855 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.856 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.857 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.857 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_autoscaler", 2024-04-25T13:23:10.857 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.857 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.857 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.857 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.857 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.857 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.858 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.858 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.858 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.858 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.858 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.858 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.858 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.858 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.858 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.859 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.860 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.860 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.860 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.860 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.860 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.860 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.860 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.860 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.860 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.861 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.861 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.861 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.861 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.861 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.861 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.861 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.861 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.861 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.862 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.862 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.862 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.862 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.862 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.862 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.862 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.862 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.863 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.863 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.863 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.863 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.863 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.863 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.863 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.863 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.863 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.864 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.864 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.864 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.864 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.864 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.864 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.864 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.864 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.864 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.865 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.865 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.865 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.865 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.865 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:10.865 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:10.865 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.865 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.865 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.866 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:10.866 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.866 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.866 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.866 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.866 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.866 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.866 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: "threshold": { 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threshold", 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1.0", 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.867 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.868 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "scaling threshold", 2024-04-25T13:23:10.868 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:23:10.868 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.868 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.868 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.868 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.868 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.868 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.869 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "progress", 2024-04-25T13:23:10.869 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.869 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.869 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.869 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_pg_recovery_event": { 2024-04-25T13:23:10.869 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:23:10.869 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.869 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.869 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.870 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.870 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.870 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.870 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.870 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:23:10.870 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.870 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.870 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.871 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.872 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.872 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.872 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.872 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.872 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.872 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.872 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.872 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.872 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.873 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.874 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.874 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.874 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.874 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.874 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.874 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.874 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.874 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.874 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.875 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.875 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.875 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.875 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.875 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.875 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.875 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.875 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.875 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.876 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.877 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.877 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.877 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.877 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.877 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.877 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.877 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.877 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.877 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.878 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.878 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.878 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.878 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.878 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.878 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.878 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.878 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.879 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.879 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.879 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.879 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.879 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.879 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.879 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.879 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.879 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.880 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.880 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.880 INFO:teuthology.orchestra.run.smithi072.stdout: "max_completed_events": { 2024-04-25T13:23:10.880 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_completed_events", 2024-04-25T13:23:10.880 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.880 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.880 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.880 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "50", 2024-04-25T13:23:10.880 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.881 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.881 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.881 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:23:10.881 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.881 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.881 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.881 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.881 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-25T13:23:10.881 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-25T13:23:10.882 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:10.882 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.882 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.882 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-25T13:23:10.882 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.882 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.882 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.882 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:23:10.883 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.883 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.883 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.883 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.883 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.883 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.883 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.883 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus", 2024-04-25T13:23:10.883 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.884 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.884 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.884 INFO:teuthology.orchestra.run.smithi072.stdout: "cache": { 2024-04-25T13:23:10.884 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cache", 2024-04-25T13:23:10.884 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.884 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.884 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.884 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.884 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: "exclude_perf_counters": { 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "exclude_perf_counters", 2024-04-25T13:23:10.885 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.886 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.886 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.886 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.886 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.886 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.886 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.886 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:23:10.886 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:23:10.887 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.887 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.887 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.887 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.887 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.887 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.887 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.887 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.888 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.888 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.888 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.888 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.888 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.888 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.888 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.888 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.888 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.889 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.890 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.890 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.890 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.890 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.890 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.890 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.890 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.890 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.891 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.891 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.891 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.891 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.891 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.891 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.891 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.891 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.891 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.892 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.893 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.893 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.893 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.893 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.893 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.893 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.893 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.893 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.893 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.894 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.894 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.894 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.894 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.894 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.894 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.894 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.894 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.894 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.895 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.895 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools": { 2024-04-25T13:23:10.895 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools", 2024-04-25T13:23:10.895 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.895 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.895 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.895 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.895 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.895 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.896 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.896 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.896 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.896 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.896 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.896 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.896 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:23:10.896 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:23:10.896 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.897 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.897 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.897 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "300", 2024-04-25T13:23:10.897 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.897 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.897 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.897 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.897 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.897 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.898 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.898 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.898 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_interval": { 2024-04-25T13:23:10.898 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_interval", 2024-04-25T13:23:10.898 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.898 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.898 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.898 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15.0", 2024-04-25T13:23:10.898 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-25T13:23:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-25T13:23:10.900 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.900 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.900 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.900 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-25T13:23:10.900 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.900 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.900 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.900 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:23:10.900 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "9283", 2024-04-25T13:23:10.901 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.902 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.902 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.902 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:23:10.902 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.902 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.902 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.902 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.902 INFO:teuthology.orchestra.run.smithi072.stdout: "stale_cache_strategy": { 2024-04-25T13:23:10.903 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stale_cache_strategy", 2024-04-25T13:23:10.903 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.903 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.903 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.903 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "log", 2024-04-25T13:23:10.903 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.903 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.903 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.903 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.904 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "default", 2024-04-25T13:23:10.905 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.905 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.905 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.905 INFO:teuthology.orchestra.run.smithi072.stdout: "default", 2024-04-25T13:23:10.905 INFO:teuthology.orchestra.run.smithi072.stdout: "error" 2024-04-25T13:23:10.905 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.905 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.905 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.905 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.906 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.906 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-25T13:23:10.906 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-25T13:23:10.906 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.906 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.906 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.906 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-25T13:23:10.906 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.907 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.908 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-04-25T13:23:10.908 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.908 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.908 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.908 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.908 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.908 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.908 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.908 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.909 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.910 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.910 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.910 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.910 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.910 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.910 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.910 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.910 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.911 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.911 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.911 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.911 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.911 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.911 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.911 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.911 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.911 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.912 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.912 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.912 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.912 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.912 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.912 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.912 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.912 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.912 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.913 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.913 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.913 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.913 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.913 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.913 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.913 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.913 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.913 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.914 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.914 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.914 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.914 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.914 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.914 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.914 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.915 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.915 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.915 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.915 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.915 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.915 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.915 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.915 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.915 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.916 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.916 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.916 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.916 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_snap_create": { 2024-04-25T13:23:10.916 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:23:10.916 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.916 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.916 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.916 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:23:10.917 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:23:10.918 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.918 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.918 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.918 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.918 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.918 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.918 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.918 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.918 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.919 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.919 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.919 INFO:teuthology.orchestra.run.smithi072.stdout: "trash_purge_schedule": { 2024-04-25T13:23:10.919 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "trash_purge_schedule", 2024-04-25T13:23:10.919 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.919 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.919 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.919 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.920 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.920 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.920 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.920 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.920 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.920 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.920 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.920 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.920 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "restful", 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_auth": { 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_auth", 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.921 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.922 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.922 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.922 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.922 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.922 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.922 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.922 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.922 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.922 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.923 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-25T13:23:10.923 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-25T13:23:10.923 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.923 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.923 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.923 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.923 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.923 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.924 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.924 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.924 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.924 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.924 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.924 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.924 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.924 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.925 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.925 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.925 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.925 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.925 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.925 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.925 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.925 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.925 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.926 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.926 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.926 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.926 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.926 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.926 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.926 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.926 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.928 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.929 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.929 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.929 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.929 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.929 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.929 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.929 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.929 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.929 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.930 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.930 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.930 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.930 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.930 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.930 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.930 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.930 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.930 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.931 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.932 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.932 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.932 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.932 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-25T13:23:10.932 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-25T13:23:10.932 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.932 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.932 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.932 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.933 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.933 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.933 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.933 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.933 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.933 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.933 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.933 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.933 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-25T13:23:10.934 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-25T13:23:10.934 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.934 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.934 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.934 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.934 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.934 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.934 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.934 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rgw", 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.935 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.936 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.936 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.936 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.936 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.936 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.936 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.936 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.936 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.936 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.937 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.937 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.937 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.937 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.937 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.937 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.937 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.937 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.937 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.939 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.939 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.939 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.939 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.939 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.939 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.939 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.939 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.940 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.941 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.941 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.941 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.941 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.941 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.941 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.941 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.941 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.941 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.943 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.944 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rook", 2024-04-25T13:23:10.944 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.944 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.944 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.944 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.944 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.944 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.944 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.944 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.945 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.945 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.945 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.945 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.945 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.945 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.945 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.945 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.946 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.946 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.946 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.946 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.946 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.946 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.946 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.946 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.946 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.947 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.947 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.947 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.947 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.947 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.947 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.947 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.947 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.947 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.948 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.949 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.950 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.950 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.950 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.950 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.950 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.950 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.950 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.950 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.951 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.951 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.951 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.951 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.951 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.951 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.951 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.951 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.951 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_class": { 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "storage_class", 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "local", 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.952 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.953 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.953 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:23:10.953 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.953 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.953 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.953 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.953 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.953 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.954 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "selftest", 2024-04-25T13:23:10.954 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.954 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.954 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.954 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.954 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.954 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.954 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.954 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.955 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.955 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.955 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.955 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.955 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.955 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.955 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.955 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.955 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.956 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.956 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.956 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.956 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.956 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.956 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.956 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.956 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.956 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.957 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.957 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.957 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.957 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.957 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.957 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.957 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.957 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.957 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.958 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.958 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.958 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.958 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.958 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.958 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.958 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.958 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.959 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.959 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.959 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.959 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.959 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.959 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.959 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.959 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.959 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.960 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.960 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.960 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.960 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.960 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.960 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.960 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.960 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.960 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.961 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.961 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.961 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.961 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.961 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.961 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.961 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.961 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.961 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.962 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.962 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.962 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.962 INFO:teuthology.orchestra.run.smithi072.stdout: "roption1": { 2024-04-25T13:23:10.962 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption1", 2024-04-25T13:23:10.962 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.962 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.962 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.962 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.963 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.963 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.963 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.963 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.963 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.963 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.963 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.963 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.963 INFO:teuthology.orchestra.run.smithi072.stdout: "roption2": { 2024-04-25T13:23:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption2", 2024-04-25T13:23:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "xyz", 2024-04-25T13:23:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.964 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.965 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.965 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.965 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.965 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.965 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.965 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption1": { 2024-04-25T13:23:10.965 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption1", 2024-04-25T13:23:10.965 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.965 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.966 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.966 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.966 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.966 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.966 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.966 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.966 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.966 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption2": { 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption2", 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.968 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.968 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.968 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.968 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.968 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.968 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.968 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption3": { 2024-04-25T13:23:10.968 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption3", 2024-04-25T13:23:10.968 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-25T13:23:10.969 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.969 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.969 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.969 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.969 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.969 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.969 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.969 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.969 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.970 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.970 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.970 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption4": { 2024-04-25T13:23:10.970 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption4", 2024-04-25T13:23:10.970 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.970 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.970 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.970 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.970 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.971 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.971 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.971 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.971 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.971 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.971 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.971 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.971 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption5": { 2024-04-25T13:23:10.971 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption5", 2024-04-25T13:23:10.972 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.972 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.972 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.972 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.972 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.972 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.972 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.972 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.972 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption6": { 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption6", 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:10.973 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.974 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.974 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.974 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.974 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.974 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.974 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.974 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.974 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption7": { 2024-04-25T13:23:10.974 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption7", 2024-04-25T13:23:10.975 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:10.975 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.975 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.975 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.975 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-25T13:23:10.975 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "42", 2024-04-25T13:23:10.975 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.975 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.975 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: "testkey": { 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testkey", 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.976 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.977 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.977 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.977 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.977 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.977 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.977 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.977 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.977 INFO:teuthology.orchestra.run.smithi072.stdout: "testlkey": { 2024-04-25T13:23:10.977 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testlkey", 2024-04-25T13:23:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.979 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.979 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.979 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.979 INFO:teuthology.orchestra.run.smithi072.stdout: "testnewline": { 2024-04-25T13:23:10.979 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testnewline", 2024-04-25T13:23:10.979 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.979 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.979 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:10.979 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.980 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.981 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.981 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-04-25T13:23:10.981 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.981 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.981 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.981 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_m_granularity": { 2024-04-25T13:23:10.981 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_m_granularity", 2024-04-25T13:23:10.981 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.981 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.982 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.982 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.982 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.982 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.982 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.982 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:23:10.982 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.982 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.982 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.983 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.983 INFO:teuthology.orchestra.run.smithi072.stdout: "dump_on_update": { 2024-04-25T13:23:10.983 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dump_on_update", 2024-04-25T13:23:10.983 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.983 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.983 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.983 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.983 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.983 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.984 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.985 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.985 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.985 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.985 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.985 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.985 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.985 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.985 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.985 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.987 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.987 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.987 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.987 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.987 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.987 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.987 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.987 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.987 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.988 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.988 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.988 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.988 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.988 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.988 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.988 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.988 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.988 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.989 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.990 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.990 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.990 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.990 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.990 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.990 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.990 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.990 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.992 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.992 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.992 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:10.992 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.992 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:10.992 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stats", 2024-04-25T13:23:10.992 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:10.992 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:10.992 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:10.993 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:10.993 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:10.993 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.993 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.993 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.993 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:10.993 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.993 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.993 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.994 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.994 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.994 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.994 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.994 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.994 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.994 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.994 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.995 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "status", 2024-04-25T13:23:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telegraf", 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: "address": { 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "address", 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.010 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.010 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:23:11.010 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.010 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.010 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.010 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.010 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.010 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.010 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.011 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.011 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:11.011 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:11.011 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:11.011 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.011 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.011 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15", 2024-04-25T13:23:11.011 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.011 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.012 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.014 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.014 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.014 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.014 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.014 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.014 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.014 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.014 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.014 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.015 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.016 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.016 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.016 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.016 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:11.016 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:11.016 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.016 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.016 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.016 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.017 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.018 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.018 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.018 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.018 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.018 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.018 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:11.018 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:11.018 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.019 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.019 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.019 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.019 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.019 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.019 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.019 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.019 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.019 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.020 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.020 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.020 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.020 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.020 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.020 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telemetry", 2024-04-25T13:23:11.020 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:11.020 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:11.020 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:11.021 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_basic": { 2024-04-25T13:23:11.021 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_basic", 2024-04-25T13:23:11.021 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.021 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.021 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.021 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:11.021 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.021 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.021 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.022 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:23:11.022 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.022 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.022 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.022 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.022 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_crash": { 2024-04-25T13:23:11.022 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_crash", 2024-04-25T13:23:11.022 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.022 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.023 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.023 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:11.023 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.023 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.023 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.023 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:23:11.023 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.023 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.023 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.024 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.024 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_device": { 2024-04-25T13:23:11.024 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_device", 2024-04-25T13:23:11.024 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.024 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.024 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.024 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:11.024 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.024 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_ident": { 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_ident", 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.025 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.026 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.026 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.026 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.026 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.026 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.026 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:23:11.026 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.026 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.026 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.027 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.027 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_perf": { 2024-04-25T13:23:11.027 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_perf", 2024-04-25T13:23:11.027 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.027 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.027 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.027 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.027 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.027 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: "contact": { 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "contact", 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.028 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.029 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.029 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.029 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.029 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.029 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.029 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.029 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.029 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.029 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.030 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.030 INFO:teuthology.orchestra.run.smithi072.stdout: "description": { 2024-04-25T13:23:11.030 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "description", 2024-04-25T13:23:11.030 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.030 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.030 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.030 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.030 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.030 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.031 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.031 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.031 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.031 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.031 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.031 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.031 INFO:teuthology.orchestra.run.smithi072.stdout: "device_url": { 2024-04-25T13:23:11.031 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_url", 2024-04-25T13:23:11.031 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.032 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.033 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.033 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-25T13:23:11.033 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-25T13:23:11.033 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.033 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.033 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.033 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.033 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.033 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.034 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.034 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.034 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.034 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.034 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.034 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.034 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:11.034 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:11.034 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "24", 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "8", 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.035 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.036 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.036 INFO:teuthology.orchestra.run.smithi072.stdout: "last_opt_revision": { 2024-04-25T13:23:11.036 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "last_opt_revision", 2024-04-25T13:23:11.036 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:11.036 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.036 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.036 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1", 2024-04-25T13:23:11.036 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.036 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.037 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.037 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.037 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.037 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.037 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.037 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.037 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard": { 2024-04-25T13:23:11.037 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard", 2024-04-25T13:23:11.037 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.038 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.038 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.038 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.038 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.038 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.038 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.038 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.038 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.038 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard_description": { 2024-04-25T13:23:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard_description", 2024-04-25T13:23:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:11.041 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.041 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.041 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.041 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.041 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.041 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.041 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.041 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.041 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.042 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.043 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:11.043 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:11.043 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.043 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.043 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.043 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.043 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.043 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.043 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.044 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.045 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:11.045 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.045 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.045 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.045 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.045 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.045 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.045 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.045 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.046 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.046 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.046 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.046 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.046 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.046 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.046 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.046 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:11.047 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:11.047 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.047 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.047 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.047 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.047 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.047 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.047 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.047 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.048 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.048 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.048 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.048 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.048 INFO:teuthology.orchestra.run.smithi072.stdout: "organization": { 2024-04-25T13:23:11.048 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "organization", 2024-04-25T13:23:11.048 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.048 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.048 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.049 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.049 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.049 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.049 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.049 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.049 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.049 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.049 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.049 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.050 INFO:teuthology.orchestra.run.smithi072.stdout: "proxy": { 2024-04-25T13:23:11.050 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "proxy", 2024-04-25T13:23:11.050 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.050 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.050 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.050 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.050 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.050 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.050 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.051 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.051 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.051 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.051 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.051 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.051 INFO:teuthology.orchestra.run.smithi072.stdout: "url": { 2024-04-25T13:23:11.051 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url", 2024-04-25T13:23:11.051 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.051 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.052 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.053 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.053 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.053 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.053 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "test_orchestrator", 2024-04-25T13:23:11.053 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:11.053 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:11.053 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:11.053 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:11.053 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:11.054 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.054 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.054 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.054 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.054 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.054 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.054 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.054 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.054 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.055 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.056 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:11.056 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:11.056 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.056 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.056 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.056 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.056 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.056 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.056 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.057 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.058 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:11.058 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.058 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.058 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.058 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.058 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.058 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.058 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.058 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.059 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.059 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.059 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.059 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.059 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.059 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.059 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.059 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:11.059 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:11.060 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.060 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.060 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.060 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.060 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.060 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.060 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.060 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.060 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:11.062 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:11.062 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:11.062 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.062 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.062 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.062 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.062 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.062 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.062 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.063 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.063 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.063 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.063 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.063 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.063 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.063 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.063 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.063 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.064 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.065 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.065 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.065 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.065 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.065 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.065 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.065 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.065 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:11.065 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:11.066 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.066 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.066 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.066 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:11.066 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.066 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.066 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.066 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.066 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.067 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.067 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.067 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.067 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.067 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.067 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.067 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.067 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.067 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.068 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.068 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:11.068 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:11.068 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.068 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.068 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.068 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.068 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.068 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.069 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.069 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.069 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.069 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.069 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.069 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.069 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_clones": { 2024-04-25T13:23:11.069 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_clones", 2024-04-25T13:23:11.069 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:11.070 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.070 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.070 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4", 2024-04-25T13:23:11.070 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.070 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.070 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.070 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:23:11.070 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.070 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.071 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.071 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.071 INFO:teuthology.orchestra.run.smithi072.stdout: "periodic_async_work": { 2024-04-25T13:23:11.071 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "periodic_async_work", 2024-04-25T13:23:11.071 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.071 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.071 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.071 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.071 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Periodically check for async work", 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_delay": { 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:23:11.072 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:11.073 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.073 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.073 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-25T13:23:11.073 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.073 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.073 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.073 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:23:11.073 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.074 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.074 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.074 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.074 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:23:11.074 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:23:11.074 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.074 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.074 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.074 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.075 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.076 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.076 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix", 2024-04-25T13:23:11.076 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-25T13:23:11.076 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-25T13:23:11.076 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-25T13:23:11.076 INFO:teuthology.orchestra.run.smithi072.stdout: "discovery_interval": { 2024-04-25T13:23:11.076 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "discovery_interval", 2024-04-25T13:23:11.076 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-25T13:23:11.076 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.077 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.077 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "100", 2024-04-25T13:23:11.077 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.077 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.077 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.077 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.077 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.077 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.077 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.078 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.078 INFO:teuthology.orchestra.run.smithi072.stdout: "identifier": { 2024-04-25T13:23:11.078 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "identifier", 2024-04-25T13:23:11.078 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.078 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.078 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.078 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.078 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.078 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-25T13:23:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-25T13:23:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.080 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.081 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.081 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-25T13:23:11.081 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-25T13:23:11.081 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.081 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.081 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.081 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.081 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.081 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.082 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.082 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.082 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.082 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.082 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.082 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.082 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.082 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.083 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.084 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.084 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.084 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.084 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.084 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.084 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.084 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.084 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.085 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.085 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-25T13:23:11.085 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-25T13:23:11.085 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.085 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.085 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.085 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-25T13:23:11.085 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.085 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.086 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-25T13:23:11.086 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-25T13:23:11.086 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-25T13:23:11.086 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-25T13:23:11.086 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-25T13:23:11.086 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-25T13:23:11.086 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-25T13:23:11.086 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.086 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.087 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.087 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.087 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.087 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.087 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-25T13:23:11.087 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-25T13:23:11.087 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-25T13:23:11.087 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.087 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-25T13:23:11.088 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-25T13:23:11.088 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.088 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.088 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.088 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.088 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.088 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.088 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.088 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.089 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_host": { 2024-04-25T13:23:11.089 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_host", 2024-04-25T13:23:11.089 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.089 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.089 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.089 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-25T13:23:11.089 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.089 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.089 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.090 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.090 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.090 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.090 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.090 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.090 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_port": { 2024-04-25T13:23:11.090 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_port", 2024-04-25T13:23:11.090 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-25T13:23:11.090 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.091 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.091 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10051", 2024-04-25T13:23:11.091 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.091 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.091 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.091 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.091 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.091 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.092 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.092 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.092 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_sender": { 2024-04-25T13:23:11.092 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_sender", 2024-04-25T13:23:11.092 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-25T13:23:11.092 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-25T13:23:11.092 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-25T13:23:11.092 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:23:11.093 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-25T13:23:11.093 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-25T13:23:11.093 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-25T13:23:11.093 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-25T13:23:11.093 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-25T13:23:11.093 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-25T13:23:11.093 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-25T13:23:11.093 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.093 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.094 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.094 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.094 INFO:teuthology.orchestra.run.smithi072.stdout: "services": { 2024-04-25T13:23:11.094 INFO:teuthology.orchestra.run.smithi072.stdout: "dashboard": "https://172.21.15.72:8443/" 2024-04-25T13:23:11.094 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.094 INFO:teuthology.orchestra.run.smithi072.stdout: "always_on_modules": { 2024-04-25T13:23:11.094 INFO:teuthology.orchestra.run.smithi072.stdout: "octopus": [ 2024-04-25T13:23:11.094 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-25T13:23:11.094 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.095 INFO:teuthology.orchestra.run.smithi072.stdout: "pacific": [ 2024-04-25T13:23:11.096 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-25T13:23:11.096 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-25T13:23:11.096 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-25T13:23:11.096 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-25T13:23:11.096 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-25T13:23:11.096 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-25T13:23:11.096 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-25T13:23:11.096 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: "quincy": [ 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-25T13:23:11.097 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-25T13:23:11.098 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-25T13:23:11.098 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-25T13:23:11.098 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-25T13:23:11.098 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-25T13:23:11.098 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.098 INFO:teuthology.orchestra.run.smithi072.stdout: "reef": [ 2024-04-25T13:23:11.098 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-25T13:23:11.098 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-25T13:23:11.098 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: "squid": [ 2024-04-25T13:23:11.099 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-25T13:23:11.100 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-25T13:23:11.100 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-25T13:23:11.100 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-25T13:23:11.100 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-25T13:23:11.100 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-25T13:23:11.100 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-25T13:23:11.100 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-25T13:23:11.100 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-25T13:23:11.100 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-25T13:23:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: "last_failure_osd_epoch": 91, 2024-04-25T13:23:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: "active_clients": [ 2024-04-25T13:23:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-04-25T13:23:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-25T13:23:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-25T13:23:11.102 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-25T13:23:11.102 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 340662568 2024-04-25T13:23:11.102 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.102 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:11.102 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.102 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.102 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "libcephsqlite", 2024-04-25T13:23:11.102 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-25T13:23:11.102 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.103 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-25T13:23:11.103 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-25T13:23:11.103 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 1475379906 2024-04-25T13:23:11.103 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.103 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:11.103 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.103 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.103 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-04-25T13:23:11.103 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 875838344 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-04-25T13:23:11.104 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-25T13:23:11.105 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.105 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-25T13:23:11.105 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-25T13:23:11.105 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3743305856 2024-04-25T13:23:11.105 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.105 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:11.105 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.105 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.105 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-04-25T13:23:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-25T13:23:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-25T13:23:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-25T13:23:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 4121415643 2024-04-25T13:23:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3473444053 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:11.107 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-25T13:23:11.108 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:23:11.109 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:09 smithi164 ceph-mon[32418]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:23:11.109 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:09 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1989606079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:23:11.109 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:09 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/314884638' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:23:11.109 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:10 smithi164 ceph-mon[32418]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:23:11.110 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:09 smithi193 ceph-mon[32743]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:23:11.110 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:09 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1989606079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:23:11.110 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:09 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/314884638' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:23:11.110 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:10 smithi193 ceph-mon[32743]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:23:11.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f65b40a3d20 msgr2=0x7f65b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:11.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f65b40a3d20 0x7f65b40a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006fd0 tx=0x7f65b8008040 comp rx=0 tx=0).stop 2024-04-25T13:23:11.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4107270 msgr2=0x7f65c419e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:11.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4107270 0x7f65c419e980 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f65b0031a90 tx=0x7f65b0004270 comp rx=0 tx=0).stop 2024-04-25T13:23:11.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 shutdown_connections 2024-04-25T13:23:11.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f65b40a3d20 0x7f65b40a61e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:11.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65c41985f0 0x7f65c4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:11.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c4107270 0x7f65c419e980 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:11.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 --2- 172.21.15.72:0/314884638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c41021a0 0x7f65c419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:11.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 >> 172.21.15.72:0/314884638 conn(0x7f65c40fd9e0 msgr2=0x7f65c40fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:11.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 shutdown_connections 2024-04-25T13:23:11.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:08.893+0000 7f65cab19640 1 -- 172.21.15.72:0/314884638 wait complete. 2024-04-25T13:23:11.119 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T13:23:11.120 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T13:23:11.120 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:09 smithi072 ceph-mon[29199]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:23:11.120 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:09 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1989606079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:23:11.121 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:09 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/314884638' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:23:11.121 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:10 smithi072 ceph-mon[29199]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:23:11.370 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:23:11.709 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:11 smithi072 ceph-mon[29199]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T13:23:11.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:11 smithi193 ceph-mon[32743]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T13:23:11.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:11 smithi164 ceph-mon[32418]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T13:23:13.119 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:23:13.119 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:13.119 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 0, 2024-04-25T13:23:13.120 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2, 2024-04-25T13:23:13.120 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:23:13.120 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 0, 2024-04-25T13:23:13.120 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-04-25T13:23:13.120 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-04-25T13:23:13.120 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.001167320, 2024-04-25T13:23:13.120 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:23:13.120 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000174465, 2024-04-25T13:23:13.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:13.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:13.121 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:13.121 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:13.121 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:13.121 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:13.121 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:13.122 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:13.122 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:13.122 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.122 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:23:13.122 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:23:13.122 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.122 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:13.122 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 6, 2024-04-25T13:23:13.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 153, 2024-04-25T13:23:13.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 3360, 2024-04-25T13:23:13.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 183232, 2024-04-25T13:23:13.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-04-25T13:23:13.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:13.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.014373439, 2024-04-25T13:23:13.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.010172920, 2024-04-25T13:23:13.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.002073251, 2024-04-25T13:23:13.124 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:13.124 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:13.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 151, 2024-04-25T13:23:13.124 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.013066829, 2024-04-25T13:23:13.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000086535 2024-04-25T13:23:13.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.124 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:13.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:13.125 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:13.125 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:13.125 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T13:23:13.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 183232 2024-04-25T13:23:13.125 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.125 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:13.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 210, 2024-04-25T13:23:13.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 199, 2024-04-25T13:23:13.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1586656, 2024-04-25T13:23:13.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 122144, 2024-04-25T13:23:13.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-04-25T13:23:13.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-04-25T13:23:13.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.042594113, 2024-04-25T13:23:13.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.013446614, 2024-04-25T13:23:13.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.034285735, 2024-04-25T13:23:13.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.004315158, 2024-04-25T13:23:13.127 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:13.127 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 198, 2024-04-25T13:23:13.127 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.011261255, 2024-04-25T13:23:13.127 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000056875 2024-04-25T13:23:13.127 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.127 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:13.127 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:13.127 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:13.127 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:13.128 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.128 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1586656, 2024-04-25T13:23:13.128 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 122144 2024-04-25T13:23:13.128 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.128 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-04-25T13:23:13.128 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-04-25T13:23:13.128 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-04-25T13:23:13.128 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.128 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-04-25T13:23:13.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:13.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:13.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:13.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:13.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:13.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:13.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:13.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:13.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:13.130 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:13.130 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:13.130 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:13.130 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:13.130 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:13.130 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:13.130 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:13.130 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:13.131 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:13.131 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:13.131 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:13.131 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:13.131 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:13.131 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:13.131 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:13.131 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:13.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:13.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:13.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:13.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:13.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:13.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:13.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-04-25T13:23:13.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:13.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:13.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:13.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:13.133 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 148933, 2024-04-25T13:23:13.133 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 30, 2024-04-25T13:23:13.133 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:23:13.133 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-04-25T13:23:13.133 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-04-25T13:23:13.133 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-04-25T13:23:13.134 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:13.134 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:13.134 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:13.134 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:13.134 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-04-25T13:23:13.134 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-04-25T13:23:13.134 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:13.134 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:13.135 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:13.135 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-04-25T13:23:13.135 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-04-25T13:23:13.135 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-04-25T13:23:13.135 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:13.135 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-04-25T13:23:13.135 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:13.135 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-04-25T13:23:13.135 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:13.136 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:13.136 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-25T13:23:13.136 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-25T13:23:13.136 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-25T13:23:13.136 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 212, 2024-04-25T13:23:13.136 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1557043, 2024-04-25T13:23:13.136 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:13.136 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 212, 2024-04-25T13:23:13.136 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-25T13:23:13.137 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-25T13:23:13.137 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 212, 2024-04-25T13:23:13.137 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1557043, 2024-04-25T13:23:13.137 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-25T13:23:13.137 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:13.137 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:13.137 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:13.137 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:13.138 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:13.138 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:23:13.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:13 smithi193 ceph-mon[32743]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T13:23:13.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:13 smithi072 ceph-mon[29199]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T13:23:13.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:13 smithi164 ceph-mon[32418]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T13:23:13.836 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T13:23:13.837 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T13:23:13.837 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-25T13:23:13.837 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-25T13:23:13.837 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T13:23:13.837 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T13:23:13.837 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T13:23:13.837 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-25T13:23:13.837 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T13:23:13.838 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T13:23:14.081 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:23:15.700 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:15 smithi072 ceph-mon[29199]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T13:23:15.717 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:23:15.718 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:15.718 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 0, 2024-04-25T13:23:15.718 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2, 2024-04-25T13:23:15.718 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:23:15.718 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 0, 2024-04-25T13:23:15.718 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-04-25T13:23:15.718 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-04-25T13:23:15.718 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.001321838, 2024-04-25T13:23:15.718 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:23:15.719 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000258967, 2024-04-25T13:23:15.719 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:15.719 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:15.719 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:15.719 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:15.719 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:15.719 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.719 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:15.719 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 215, 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 204, 2024-04-25T13:23:15.720 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1588016, 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 125344, 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.044133544, 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.014322293, 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.034046944, 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.004442534, 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 203, 2024-04-25T13:23:15.721 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.010872738, 2024-04-25T13:23:15.722 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000053560 2024-04-25T13:23:15.722 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.722 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:15.722 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:15.722 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:15.722 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:15.722 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.722 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1588016, 2024-04-25T13:23:15.723 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 125344 2024-04-25T13:23:15.723 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.723 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:15.723 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 6, 2024-04-25T13:23:15.723 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 158, 2024-04-25T13:23:15.723 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 3360, 2024-04-25T13:23:15.723 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 184112, 2024-04-25T13:23:15.723 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-04-25T13:23:15.723 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:15.724 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.014995680, 2024-04-25T13:23:15.724 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.009916852, 2024-04-25T13:23:15.724 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.002603623, 2024-04-25T13:23:15.724 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:15.724 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:15.724 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 156, 2024-04-25T13:23:15.724 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.014319130, 2024-04-25T13:23:15.724 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000091789 2024-04-25T13:23:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T13:23:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 184112 2024-04-25T13:23:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.726 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-04-25T13:23:15.726 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-04-25T13:23:15.726 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-04-25T13:23:15.726 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.726 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-04-25T13:23:15.726 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:15.726 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:15.726 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:15.726 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:15.727 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:15.727 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:15.727 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:15.727 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:15.727 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:15.727 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:15.727 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:15.727 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:15.727 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:15.728 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:15.728 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:15.728 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:15.728 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:15.728 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:15.728 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:15.728 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:15.728 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:15.729 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:15.729 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:15.729 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:15.729 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:15.729 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:15.729 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:15.729 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:15.729 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:15.729 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:15.730 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:15.730 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-04-25T13:23:15.730 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:15.730 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:15.730 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:15.730 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:15.730 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:23:15.730 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 30, 2024-04-25T13:23:15.731 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:23:15.731 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-04-25T13:23:15.731 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-04-25T13:23:15.731 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-04-25T13:23:15.731 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:15.731 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:15.731 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:15.731 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:15.731 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-04-25T13:23:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-04-25T13:23:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-04-25T13:23:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-04-25T13:23:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-04-25T13:23:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 217, 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1557693, 2024-04-25T13:23:15.733 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:15.734 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 217, 2024-04-25T13:23:15.734 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-25T13:23:15.734 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-25T13:23:15.734 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 217, 2024-04-25T13:23:15.734 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1557693, 2024-04-25T13:23:15.734 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-25T13:23:15.734 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:15.734 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:15.734 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:15.735 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:15.735 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:15.735 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:23:15.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:15 smithi193 ceph-mon[32743]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T13:23:15.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:15 smithi164 ceph-mon[32418]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T13:23:16.547 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T13:23:16.548 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T13:23:16.548 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T13:23:16.548 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T13:23:16.548 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T13:23:16.548 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:23:16.548 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:23:16.548 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-25T13:23:16.548 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T13:23:16.549 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T13:23:16.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:16 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:16.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:16 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:16.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:16 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:17.062 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:23:17.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:17 smithi193 ceph-mon[32743]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T13:23:17.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:17 smithi193 ceph-mon[32743]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:23:17.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:17 smithi072 ceph-mon[29199]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T13:23:17.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:17 smithi072 ceph-mon[29199]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:23:17.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:17 smithi164 ceph-mon[32418]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T13:23:17.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:17 smithi164 ceph-mon[32418]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:23:18.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 249 B/s wr, 11 op/s 2024-04-25T13:23:18.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:18.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:18.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:18.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:18.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-04-25T13:23:18.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-25T13:23:18.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T13:23:18.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:18 smithi072 ceph-mon[29199]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T13:23:18.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 249 B/s wr, 11 op/s 2024-04-25T13:23:18.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:18.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:18.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:18.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:18.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-04-25T13:23:18.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-25T13:23:18.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:18.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T13:23:18.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:18 smithi193 ceph-mon[32743]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T13:23:19.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 249 B/s wr, 11 op/s 2024-04-25T13:23:19.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:19.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:19.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:19.057 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:19.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:19.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:19.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:19.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:19.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-04-25T13:23:19.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-25T13:23:19.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:19.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:19.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:19.059 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:19.059 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T13:23:19.059 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:18 smithi164 ceph-mon[32418]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T13:23:19.235 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:23:19.235 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 0, 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2, 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 0, 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.001029158, 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000137169, 2024-04-25T13:23:19.236 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:19.237 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:19.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:19.237 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:19.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:19.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.237 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:19.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:19.237 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:19.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:19.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.238 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:23:19.238 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:23:19.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.238 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:19.238 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 216, 2024-04-25T13:23:19.238 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 203, 2024-04-25T13:23:19.238 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1590240, 2024-04-25T13:23:19.238 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 123872, 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.042740642, 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.013723903, 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.033060563, 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.004268134, 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 203, 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.012248054, 2024-04-25T13:23:19.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000060335 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1590240, 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 123872 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.240 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 2, 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 153, 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 183136, 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.013389617, 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.009532210, 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.001589915, 2024-04-25T13:23:19.241 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:19.242 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:19.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 150, 2024-04-25T13:23:19.242 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.016242706, 2024-04-25T13:23:19.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000108284 2024-04-25T13:23:19.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.242 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:19.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:19.242 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:19.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:19.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.243 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:23:19.243 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 183136 2024-04-25T13:23:19.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.243 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-04-25T13:23:19.243 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-04-25T13:23:19.243 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-04-25T13:23:19.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.243 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-04-25T13:23:19.244 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:19.244 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:19.244 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:19.244 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:19.244 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:19.244 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:19.244 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:19.244 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:19.244 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:19.245 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:19.245 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:19.245 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:19.245 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:19.245 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:19.245 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:19.245 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:19.245 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:19.245 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:19.246 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:19.246 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:19.246 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:19.246 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:19.246 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:19.246 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:19.246 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:19.246 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:19.246 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:19.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:19.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:19.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:19.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:19.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-04-25T13:23:19.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:19.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:19.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:19.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:19.248 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:23:19.248 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 31, 2024-04-25T13:23:19.248 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:23:19.248 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-04-25T13:23:19.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-04-25T13:23:19.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-04-25T13:23:19.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:19.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:19.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:19.249 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:19.249 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-04-25T13:23:19.249 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-04-25T13:23:19.249 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:19.249 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:19.249 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:19.249 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-04-25T13:23:19.249 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-04-25T13:23:19.249 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-04-25T13:23:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-04-25T13:23:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-04-25T13:23:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-25T13:23:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-25T13:23:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 214, 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1557303, 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 214, 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 214, 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1557303, 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-25T13:23:19.251 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:19.252 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:19.252 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:19.252 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:19.252 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:19.252 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:23:19.927 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T13:23:19.927 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T13:23:19.927 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-25T13:23:19.928 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-25T13:23:19.928 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T13:23:19.928 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T13:23:19.928 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T13:23:19.928 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-25T13:23:19.928 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T13:23:19.928 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T13:23:20.196 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:23:21.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:23:21.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:21.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:21.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:21.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:21.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:21 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:21.483 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:23:21.483 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:21.484 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:21.484 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.484 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.484 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.484 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.484 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:21.484 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:21.484 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:21 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:21.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:23:21.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:21.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:21.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:21.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:21.558 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:21 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' 2024-04-25T13:23:21.883 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-25T13:23:21.883 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:21.883 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 221, 2024-04-25T13:23:21.883 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 208, 2024-04-25T13:23:21.883 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1591600, 2024-04-25T13:23:21.883 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 127072, 2024-04-25T13:23:21.884 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-04-25T13:23:21.884 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-04-25T13:23:21.884 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.046663847, 2024-04-25T13:23:21.884 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.015103250, 2024-04-25T13:23:21.884 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.035763524, 2024-04-25T13:23:21.884 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.004214649, 2024-04-25T13:23:21.884 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:21.884 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 208, 2024-04-25T13:23:21.884 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.010883772, 2024-04-25T13:23:21.885 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000052325 2024-04-25T13:23:21.885 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.885 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:21.885 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:21.885 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:21.885 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:21.885 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.885 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1591600, 2024-04-25T13:23:21.885 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 127072 2024-04-25T13:23:21.886 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.886 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:21.886 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 2, 2024-04-25T13:23:21.886 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 158, 2024-04-25T13:23:21.886 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:23:21.886 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 184016, 2024-04-25T13:23:21.886 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-04-25T13:23:21.886 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:21.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.013987940, 2024-04-25T13:23:21.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.009553721, 2024-04-25T13:23:21.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.001906993, 2024-04-25T13:23:21.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:21.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:21.887 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 155, 2024-04-25T13:23:21.887 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.018190278, 2024-04-25T13:23:21.887 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000117356 2024-04-25T13:23:21.887 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.888 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:21.888 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:21.888 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:21.888 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:21.888 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.888 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:23:21.888 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 184016 2024-04-25T13:23:21.888 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.888 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:21.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 0, 2024-04-25T13:23:21.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2, 2024-04-25T13:23:21.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:23:21.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 0, 2024-04-25T13:23:21.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-04-25T13:23:21.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-04-25T13:23:21.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.000369362, 2024-04-25T13:23:21.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:23:21.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000289125, 2024-04-25T13:23:21.890 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:21.890 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:21.890 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:21.890 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:21.890 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:21.890 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.890 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:21.890 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:21.890 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:21.891 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:21.891 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:23:21.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:23:21.891 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.891 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-04-25T13:23:21.891 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-04-25T13:23:21.891 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-04-25T13:23:21.892 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.892 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-04-25T13:23:21.892 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:21.892 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:21.892 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:21.892 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:21.892 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:21.892 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:21.892 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:21.893 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:21.893 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:21.893 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:21.893 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:21.893 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:21.893 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:21.893 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:21.893 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:21.893 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:21.894 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:21.894 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:21.894 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:21.894 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:21.894 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:21.894 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:21.894 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:21.894 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:21.894 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:21.895 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:21.895 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:21.895 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:21.895 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:21.895 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:21.895 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:21.895 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-04-25T13:23:21.895 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:21.895 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:21.896 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:21.896 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:21.896 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 150008, 2024-04-25T13:23:21.896 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 31, 2024-04-25T13:23:21.896 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:23:21.896 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-04-25T13:23:21.896 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-04-25T13:23:21.896 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-04-25T13:23:21.896 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:21.897 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:21.897 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:21.897 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:21.897 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-04-25T13:23:21.897 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-04-25T13:23:21.897 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:21.897 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:21.897 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:21.897 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-04-25T13:23:21.898 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-04-25T13:23:21.898 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-04-25T13:23:21.898 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:21.898 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-04-25T13:23:21.898 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:21.898 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-04-25T13:23:21.898 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-25T13:23:21.898 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:21.898 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-25T13:23:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-25T13:23:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-25T13:23:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 219, 2024-04-25T13:23:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1557953, 2024-04-25T13:23:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 219, 2024-04-25T13:23:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-25T13:23:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-25T13:23:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 219, 2024-04-25T13:23:21.900 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1557953, 2024-04-25T13:23:21.900 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-25T13:23:21.900 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-25T13:23:21.900 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-25T13:23:21.900 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-25T13:23:21.900 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:21.900 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-25T13:23:21.900 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-25T13:23:22.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:22 smithi193 ceph-mon[32743]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-04-25T13:23:22.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:22 smithi193 ceph-mon[32743]: Unable to set osd_memory_target on smithi072 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-25T13:23:22.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:22 smithi072 ceph-mon[29199]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-04-25T13:23:22.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:22 smithi072 ceph-mon[29199]: Unable to set osd_memory_target on smithi072 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-25T13:23:22.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:22 smithi164 ceph-mon[32418]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-04-25T13:23:22.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:22 smithi164 ceph-mon[32418]: Unable to set osd_memory_target on smithi072 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-25T13:23:22.579 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T13:23:22.579 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T13:23:22.579 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T13:23:22.580 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T13:23:22.580 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T13:23:22.580 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:23:22.580 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:23:22.580 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-25T13:23:22.580 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T13:23:22.580 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T13:23:22.831 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:23:23.214 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:23 smithi164 ceph-mon[32418]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:23:23.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:23 smithi193 ceph-mon[32743]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:23:23.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:23 smithi072 ceph-mon[29199]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:23:24.390 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T13:23:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1424, 2024-04-25T13:23:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 629, 2024-04-25T13:23:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 2107464, 2024-04-25T13:23:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 331090, 2024-04-25T13:23:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 16, 2024-04-25T13:23:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2024-04-25T13:23:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.179707890, 2024-04-25T13:23:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.041801136, 2024-04-25T13:23:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.087832316, 2024-04-25T13:23:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.058768800, 2024-04-25T13:23:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 623, 2024-04-25T13:23:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.561175354, 2024-04-25T13:23:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000900763 2024-04-25T13:23:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1200, 2024-04-25T13:23:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000242053, 2024-04-25T13:23:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000201 2024-04-25T13:23:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 2068752, 2024-04-25T13:23:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 252000 2024-04-25T13:23:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1258, 2024-04-25T13:23:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 637, 2024-04-25T13:23:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 508667, 2024-04-25T13:23:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 587197, 2024-04-25T13:23:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 10, 2024-04-25T13:23:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 6, 2024-04-25T13:23:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.154212056, 2024-04-25T13:23:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.038425385, 2024-04-25T13:23:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.049633004, 2024-04-25T13:23:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.059755184, 2024-04-25T13:23:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 640, 2024-04-25T13:23:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.703056923, 2024-04-25T13:23:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001098526 2024-04-25T13:23:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1346, 2024-04-25T13:23:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000291984, 2024-04-25T13:23:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000216 2024-04-25T13:23:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T13:23:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 440448 2024-04-25T13:23:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 841, 2024-04-25T13:23:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 583, 2024-04-25T13:23:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 308123, 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 299026, 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 11, 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.090024019, 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.024075639, 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.032161187, 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.029640073, 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 581, 2024-04-25T13:23:24.397 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.316370187, 2024-04-25T13:23:24.398 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000544527 2024-04-25T13:23:24.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.398 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:24.398 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 790, 2024-04-25T13:23:24.398 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000159873, 2024-04-25T13:23:24.398 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000202 2024-04-25T13:23:24.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.398 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 42560, 2024-04-25T13:23:24.399 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 1936 2024-04-25T13:23:24.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.399 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-25T13:23:24.399 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-25T13:23:24.399 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-25T13:23:24.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.399 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-25T13:23:24.399 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T13:23:24.399 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T13:23:24.400 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 213, 2024-04-25T13:23:24.400 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.044030910, 2024-04-25T13:23:24.400 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000206717 2024-04-25T13:23:24.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:24.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.400 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-25T13:23:24.400 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T13:23:24.400 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T13:23:24.400 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-04-25T13:23:24.401 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000012603, 2024-04-25T13:23:24.401 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000006301 2024-04-25T13:23:24.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:24.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.401 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-25T13:23:24.401 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 146, 2024-04-25T13:23:24.401 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 146, 2024-04-25T13:23:24.401 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-25T13:23:24.401 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 146, 2024-04-25T13:23:24.402 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.129343474, 2024-04-25T13:23:24.402 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000885914 2024-04-25T13:23:24.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.402 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-25T13:23:24.402 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-25T13:23:24.402 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-25T13:23:24.402 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-25T13:23:24.402 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 12, 2024-04-25T13:23:24.402 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-25T13:23:24.403 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-25T13:23:24.403 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 25, 2024-04-25T13:23:24.403 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 2, 2024-04-25T13:23:24.403 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 23, 2024-04-25T13:23:24.403 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-25T13:23:24.403 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 22, 2024-04-25T13:23:24.403 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-25T13:23:24.403 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 2, 2024-04-25T13:23:24.403 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 2, 2024-04-25T13:23:24.404 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 2, 2024-04-25T13:23:24.404 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 182, 2024-04-25T13:23:24.404 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 173, 2024-04-25T13:23:24.404 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-25T13:23:24.404 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-25T13:23:24.404 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:23:24.404 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-25T13:23:24.404 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-25T13:23:24.404 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-25T13:23:24.405 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-25T13:23:24.405 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-25T13:23:24.405 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-25T13:23:24.405 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-25T13:23:24.405 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-25T13:23:24.405 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-25T13:23:24.405 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:23:24.405 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-25T13:23:24.405 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-04-25T13:23:24.406 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-04-25T13:23:24.406 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 6, 2024-04-25T13:23:24.406 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:23:24.406 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-25T13:23:24.406 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:23:24.406 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:23:24.406 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:23:24.406 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:23:24.407 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:23:24.407 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-25T13:23:24.407 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:23:24.407 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T13:23:24.407 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 5, 2024-04-25T13:23:24.407 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:23:24.407 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:23:24.407 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:23:24.408 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 14, 2024-04-25T13:23:24.408 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 2, 2024-04-25T13:23:24.408 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-25T13:23:24.408 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 16 2024-04-25T13:23:24.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.408 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-25T13:23:24.408 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 4, 2024-04-25T13:23:24.408 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-25T13:23:24.408 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-25T13:23:24.409 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-04-25T13:23:24.409 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 12, 2024-04-25T13:23:24.409 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-25T13:23:24.409 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-25T13:23:24.409 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:23:24.409 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-25T13:23:24.409 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-25T13:23:24.409 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-25T13:23:24.410 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-25T13:23:24.410 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-25T13:23:24.410 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:23:24.410 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:23:24.410 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-25T13:23:24.410 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-25T13:23:24.410 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:23:24.411 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:23:24.411 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:23:24.411 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-25T13:23:24.411 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-25T13:23:24.411 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:23:24.411 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-25T13:23:24.411 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-25T13:23:24.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.411 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-25T13:23:24.412 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-25T13:23:24.412 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 103, 2024-04-25T13:23:24.412 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-25T13:23:24.412 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-25T13:23:24.412 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 103, 2024-04-25T13:23:24.412 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-25T13:23:24.412 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-25T13:23:24.412 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 2, 2024-04-25T13:23:24.413 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-25T13:23:24.413 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-25T13:23:24.413 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-25T13:23:24.413 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-25T13:23:24.413 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-25T13:23:24.413 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-25T13:23:24.413 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-25T13:23:24.413 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4286986, 2024-04-25T13:23:24.413 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-04-25T13:23:24.414 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-25T13:23:24.414 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 84, 2024-04-25T13:23:24.414 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.154515445, 2024-04-25T13:23:24.414 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001839469 2024-04-25T13:23:24.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.414 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-04-25T13:23:24.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.414 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-25T13:23:24.415 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 23, 2024-04-25T13:23:24.415 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 23, 2024-04-25T13:23:24.415 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 0, 2024-04-25T13:23:24.415 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 18, 2024-04-25T13:23:24.415 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 18, 2024-04-25T13:23:24.415 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 0, 2024-04-25T13:23:24.415 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 25, 2024-04-25T13:23:24.415 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 25, 2024-04-25T13:23:24.416 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 0, 2024-04-25T13:23:24.416 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 2, 2024-04-25T13:23:24.416 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 16, 2024-04-25T13:23:24.416 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 14, 2024-04-25T13:23:24.416 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 55168, 2024-04-25T13:23:24.416 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 231800 2024-04-25T13:23:24.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.416 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-25T13:23:24.417 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 187, 2024-04-25T13:23:24.417 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-25T13:23:24.417 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 146, 2024-04-25T13:23:24.417 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 35, 2024-04-25T13:23:24.417 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-25T13:23:24.417 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-25T13:23:24.417 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4, 2024-04-25T13:23:24.417 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.003556950, 2024-04-25T13:23:24.417 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000889237 2024-04-25T13:23:24.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.418 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-25T13:23:24.418 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 98, 2024-04-25T13:23:24.418 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.021765542, 2024-04-25T13:23:24.418 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000222097 2024-04-25T13:23:24.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.418 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-25T13:23:24.418 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.418 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.419 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.419 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-25T13:23:24.419 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.419 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.419 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.419 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-25T13:23:24.419 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-04-25T13:23:24.420 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000312193, 2024-04-25T13:23:24.420 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000156096 2024-04-25T13:23:24.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.420 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-25T13:23:24.420 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.420 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.420 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.420 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-25T13:23:24.421 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.421 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.421 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.421 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-25T13:23:24.421 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.421 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.421 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.422 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-25T13:23:24.422 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.422 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.422 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.422 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-25T13:23:24.422 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.422 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.423 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.423 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-25T13:23:24.423 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 10, 2024-04-25T13:23:24.423 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001521075, 2024-04-25T13:23:24.423 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000152107 2024-04-25T13:23:24.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.424 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-25T13:23:24.424 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 6, 2024-04-25T13:23:24.424 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.008792478, 2024-04-25T13:23:24.424 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001465413 2024-04-25T13:23:24.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.424 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-25T13:23:24.425 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.425 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.425 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.425 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-25T13:23:24.425 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 10, 2024-04-25T13:23:24.425 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.049768096, 2024-04-25T13:23:24.425 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.004976809 2024-04-25T13:23:24.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.426 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-25T13:23:24.426 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-04-25T13:23:24.426 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000295669, 2024-04-25T13:23:24.426 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000295669 2024-04-25T13:23:24.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.426 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-25T13:23:24.426 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.426 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.427 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.427 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-25T13:23:24.427 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-04-25T13:23:24.427 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002986957, 2024-04-25T13:23:24.427 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001493478 2024-04-25T13:23:24.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.427 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-25T13:23:24.428 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.428 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.428 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.428 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-25T13:23:24.428 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.428 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.428 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.429 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-25T13:23:24.429 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.429 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.429 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.429 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-25T13:23:24.429 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4, 2024-04-25T13:23:24.429 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.017001358, 2024-04-25T13:23:24.430 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.004250339 2024-04-25T13:23:24.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.430 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-25T13:23:24.430 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.430 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.430 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.430 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-25T13:23:24.431 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-04-25T13:23:24.431 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001462818, 2024-04-25T13:23:24.431 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001462818 2024-04-25T13:23:24.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.431 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-25T13:23:24.431 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.431 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.431 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.432 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-25T13:23:24.432 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.432 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.432 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.432 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-25T13:23:24.432 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.432 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.432 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.433 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-25T13:23:24.433 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 8, 2024-04-25T13:23:24.433 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.021880338, 2024-04-25T13:23:24.433 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.002735042 2024-04-25T13:23:24.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.433 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-25T13:23:24.433 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.434 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.434 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.434 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-25T13:23:24.434 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.434 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.434 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.435 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:23:24.435 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:23:24.435 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-25T13:23:24.435 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.435 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.435 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.435 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:24.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.436 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-25T13:23:24.436 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 2, 2024-04-25T13:23:24.436 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 6, 2024-04-25T13:23:24.436 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 4, 2024-04-25T13:23:24.436 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 2, 2024-04-25T13:23:24.436 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-25T13:23:24.436 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 27, 2024-04-25T13:23:24.436 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 13, 2024-04-25T13:23:24.437 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 592, 2024-04-25T13:23:24.437 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-25T13:23:24.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.437 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-25T13:23:24.437 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:24.437 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:24.437 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:24.437 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:24.437 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:24.438 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:24.438 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:24.438 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:24.438 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:24.438 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:24.438 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:24.438 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:24.438 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:24.438 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:24.439 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:24.439 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:24.439 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:24.439 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:24.439 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:24.439 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:24.439 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:24.439 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:24.439 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:24.441 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-25T13:23:24.441 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:24.441 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:24.441 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:24.441 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:24.441 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 351301, 2024-04-25T13:23:24.441 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 148, 2024-04-25T13:23:24.441 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 880, 2024-04-25T13:23:24.441 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 10, 2024-04-25T13:23:24.442 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-25T13:23:24.442 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-25T13:23:24.442 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:24.442 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:24.442 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:24.442 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:24.442 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8136, 2024-04-25T13:23:24.442 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 170, 2024-04-25T13:23:24.442 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:24.443 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:24.443 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:24.443 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-25T13:23:24.443 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 95122, 2024-04-25T13:23:24.443 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 580, 2024-04-25T13:23:24.443 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:24.443 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-25T13:23:24.443 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:24.444 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-25T13:23:24.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.444 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-25T13:23:24.444 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-25T13:23:24.444 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-25T13:23:24.444 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 646, 2024-04-25T13:23:24.444 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 107081, 2024-04-25T13:23:24.444 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 34, 2024-04-25T13:23:24.444 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 646, 2024-04-25T13:23:24.445 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-25T13:23:24.445 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 646, 2024-04-25T13:23:24.445 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 19.111681792, 2024-04-25T13:23:24.445 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.029584646 2024-04-25T13:23:24.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.445 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-25T13:23:24.445 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-25T13:23:24.445 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 646, 2024-04-25T13:23:24.446 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 680 2024-04-25T13:23:24.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.446 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 646, 2024-04-25T13:23:24.446 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 0, 2024-04-25T13:23:24.446 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 646, 2024-04-25T13:23:24.446 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-25T13:23:24.446 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-25T13:23:24.446 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 0, 2024-04-25T13:23:24.447 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 11, 2024-04-25T13:23:24.447 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 0, 2024-04-25T13:23:24.447 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 84, 2024-04-25T13:23:24.447 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 21, 2024-04-25T13:23:24.447 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-25T13:23:24.447 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-25T13:23:24.447 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-25T13:23:24.447 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-25T13:23:24.447 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 506, 2024-04-25T13:23:24.448 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-25T13:23:24.448 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-25T13:23:24.448 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-25T13:23:24.448 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 0, 2024-04-25T13:23:24.448 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 22, 2024-04-25T13:23:24.448 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:23:24.448 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-25T13:23:24.448 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:23:24.449 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-25T13:23:24.449 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-25T13:23:24.449 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-25T13:23:24.449 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:23:24.449 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-25T13:23:24.449 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:23:24.449 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-25T13:23:24.449 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-25T13:23:24.449 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-25T13:23:24.450 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-25T13:23:24.450 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-25T13:23:24.450 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-25T13:23:24.450 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-25T13:23:24.450 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-25T13:23:24.450 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-25T13:23:24.450 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-25T13:23:24.450 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-25T13:23:24.451 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-25T13:23:24.451 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-25T13:23:24.451 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-25T13:23:24.451 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-25T13:23:24.451 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-25T13:23:24.451 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-25T13:23:24.451 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 99, 2024-04-25T13:23:24.451 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-25T13:23:24.451 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 12, 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 12, 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 12, 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 32, 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 4 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-25T13:23:24.452 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-04-25T13:23:24.453 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:23:24.453 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-25T13:23:24.453 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-25T13:23:24.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.453 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-25T13:23:24.453 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-25T13:23:24.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.453 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-25T13:23:24.453 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-25T13:23:24.454 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:23:24.454 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-25T13:23:24.454 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-25T13:23:24.454 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-25T13:23:24.454 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-25T13:23:24.454 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-25T13:23:24.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.454 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:24.455 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:24.455 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T13:23:24.455 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:24.455 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 3519, 2024-04-25T13:23:24.455 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2540549, 2024-04-25T13:23:24.455 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:24.455 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 3519, 2024-04-25T13:23:24.455 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 3519, 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2540549, 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.456 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-25T13:23:24.457 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:24.457 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T13:23:24.457 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:24.457 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:24.457 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:24.457 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:24.457 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:24.457 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 646, 2024-04-25T13:23:24.457 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 107081, 2024-04-25T13:23:24.458 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 646, 2024-04-25T13:23:24.458 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 107081, 2024-04-25T13:23:24.458 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:24.458 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.458 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.458 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:24.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.458 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-25T13:23:24.459 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:24.459 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-25T13:23:24.459 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:24.459 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:24.459 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:24.459 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:24.459 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:24.459 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 646, 2024-04-25T13:23:24.459 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 646, 2024-04-25T13:23:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 646, 2024-04-25T13:23:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 646, 2024-04-25T13:23:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:23:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T13:23:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-25T13:23:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-25T13:23:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55d033888500": { 2024-04-25T13:23:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T13:23:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 103, 2024-04-25T13:23:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 92682, 2024-04-25T13:23:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 103, 2024-04-25T13:23:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 84, 2024-04-25T13:23:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 92682, 2024-04-25T13:23:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:24.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:24.465 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T13:23:25.173 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T13:23:25.173 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T13:23:25.173 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=4 2024-04-25T13:23:25.173 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T13:23:25.173 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T13:23:25.173 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T13:23:25.174 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T13:23:25.174 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-04-25T13:23:25.174 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T13:23:25.174 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T13:23:25.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:25 smithi164 ceph-mon[32418]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:23:25.423 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:23:25.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:25 smithi193 ceph-mon[32743]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:23:25.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:25 smithi072 ceph-mon[29199]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:23:27.022 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T13:23:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 745, 2024-04-25T13:23:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 747, 2024-04-25T13:23:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 197709, 2024-04-25T13:23:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 236974, 2024-04-25T13:23:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2024-04-25T13:23:27.024 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-25T13:23:27.024 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.064558117, 2024-04-25T13:23:27.024 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.025548771, 2024-04-25T13:23:27.024 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.015441166, 2024-04-25T13:23:27.024 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.017079883, 2024-04-25T13:23:27.024 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:27.024 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 745, 2024-04-25T13:23:27.024 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.023953607, 2024-04-25T13:23:27.024 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000032152 2024-04-25T13:23:27.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.025 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:27.025 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.025 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.025 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.025 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:23:27.025 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:23:27.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.026 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:27.026 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 861, 2024-04-25T13:23:27.026 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 849, 2024-04-25T13:23:27.026 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1767288, 2024-04-25T13:23:27.026 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 328783, 2024-04-25T13:23:27.026 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2024-04-25T13:23:27.026 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:27.027 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.099264665, 2024-04-25T13:23:27.027 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.029678777, 2024-04-25T13:23:27.027 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.049171886, 2024-04-25T13:23:27.027 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.025799160, 2024-04-25T13:23:27.027 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:27.027 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 848, 2024-04-25T13:23:27.027 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.030885583, 2024-04-25T13:23:27.027 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000036421 2024-04-25T13:23:27.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.028 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:27.028 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.028 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.028 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.028 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1613024, 2024-04-25T13:23:27.028 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 129392 2024-04-25T13:23:27.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.029 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:27.029 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 743, 2024-04-25T13:23:27.029 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 897, 2024-04-25T13:23:27.029 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 250992, 2024-04-25T13:23:27.029 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 674942, 2024-04-25T13:23:27.029 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2024-04-25T13:23:27.029 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-25T13:23:27.029 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.088150642, 2024-04-25T13:23:27.029 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.032927167, 2024-04-25T13:23:27.030 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.021148393, 2024-04-25T13:23:27.030 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.024034710, 2024-04-25T13:23:27.030 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:27.030 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 895, 2024-04-25T13:23:27.030 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.054070310, 2024-04-25T13:23:27.030 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000060413 2024-04-25T13:23:27.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.030 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:27.031 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.031 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.031 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.031 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T13:23:27.031 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 440576 2024-04-25T13:23:27.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.031 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-25T13:23:27.031 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-25T13:23:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-25T13:23:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-25T13:23:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T13:23:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T13:23:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3520, 2024-04-25T13:23:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.094101971, 2024-04-25T13:23:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000026733 2024-04-25T13:23:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:27.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.033 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-25T13:23:27.033 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T13:23:27.033 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T13:23:27.033 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.033 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.033 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.033 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:27.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.034 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-25T13:23:27.034 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-04-25T13:23:27.034 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-04-25T13:23:27.034 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-25T13:23:27.034 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.034 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.034 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.035 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-25T13:23:27.035 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-25T13:23:27.035 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-25T13:23:27.035 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-25T13:23:27.035 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-04-25T13:23:27.035 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-25T13:23:27.035 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-25T13:23:27.035 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 0, 2024-04-25T13:23:27.036 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-04-25T13:23:27.036 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 0, 2024-04-25T13:23:27.036 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-25T13:23:27.036 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 0, 2024-04-25T13:23:27.036 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-25T13:23:27.036 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-04-25T13:23:27.036 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-04-25T13:23:27.037 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2024-04-25T13:23:27.037 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-04-25T13:23:27.037 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-04-25T13:23:27.037 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-25T13:23:27.037 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-25T13:23:27.037 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:23:27.037 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-25T13:23:27.037 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-25T13:23:27.037 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-25T13:23:27.038 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-25T13:23:27.038 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-25T13:23:27.038 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-25T13:23:27.038 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-25T13:23:27.038 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-25T13:23:27.038 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-25T13:23:27.038 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:23:27.038 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-25T13:23:27.039 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2024-04-25T13:23:27.039 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2024-04-25T13:23:27.039 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-25T13:23:27.039 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:23:27.039 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-25T13:23:27.039 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:23:27.039 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:23:27.039 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:23:27.039 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:23:27.040 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:23:27.040 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-25T13:23:27.040 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:23:27.040 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:23:27.040 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:23:27.040 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:23:27.040 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:23:27.040 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:23:27.040 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-25T13:23:27.041 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:23:27.041 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-25T13:23:27.041 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-25T13:23:27.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.041 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-25T13:23:27.041 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-25T13:23:27.041 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-25T13:23:27.041 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-25T13:23:27.041 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-04-25T13:23:27.042 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-25T13:23:27.042 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-25T13:23:27.042 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-25T13:23:27.042 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:23:27.042 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-25T13:23:27.042 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-25T13:23:27.042 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-25T13:23:27.042 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-25T13:23:27.043 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-25T13:23:27.043 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:23:27.043 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:23:27.043 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-25T13:23:27.043 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-25T13:23:27.043 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:23:27.043 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:23:27.043 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:23:27.043 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-25T13:23:27.044 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-25T13:23:27.044 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:23:27.044 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-25T13:23:27.044 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-25T13:23:27.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.044 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-25T13:23:27.044 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-25T13:23:27.044 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-04-25T13:23:27.044 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-25T13:23:27.045 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-25T13:23:27.045 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 7, 2024-04-25T13:23:27.045 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-25T13:23:27.045 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-25T13:23:27.045 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-04-25T13:23:27.045 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-25T13:23:27.045 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-25T13:23:27.045 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-25T13:23:27.045 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-25T13:23:27.046 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-25T13:23:27.046 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-25T13:23:27.046 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-25T13:23:27.046 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4196264, 2024-04-25T13:23:27.046 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 4196123, 2024-04-25T13:23:27.046 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-25T13:23:27.046 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.046 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.046 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.047 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 7 2024-04-25T13:23:27.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.047 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-25T13:23:27.047 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 3, 2024-04-25T13:23:27.047 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 3, 2024-04-25T13:23:27.047 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 0, 2024-04-25T13:23:27.047 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 1, 2024-04-25T13:23:27.047 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 1, 2024-04-25T13:23:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 0, 2024-04-25T13:23:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 0, 2024-04-25T13:23:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 0, 2024-04-25T13:23:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 0, 2024-04-25T13:23:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-04-25T13:23:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-04-25T13:23:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-04-25T13:23:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 49336, 2024-04-25T13:23:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215416 2024-04-25T13:23:27.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.049 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-25T13:23:27.049 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-04-25T13:23:27.049 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-25T13:23:27.049 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-04-25T13:23:27.049 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-04-25T13:23:27.049 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-25T13:23:27.049 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-25T13:23:27.049 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.050 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.050 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.050 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-25T13:23:27.050 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.050 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.050 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.051 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-25T13:23:27.051 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.051 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.051 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.051 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-25T13:23:27.051 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.051 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.052 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.052 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-25T13:23:27.052 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.052 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.052 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.052 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-25T13:23:27.053 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.053 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.053 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.053 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-25T13:23:27.053 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.053 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.053 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.054 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-25T13:23:27.054 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.054 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.054 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.054 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-25T13:23:27.054 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.054 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.055 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.055 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-25T13:23:27.055 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.055 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.055 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.055 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-25T13:23:27.056 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.056 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.056 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.056 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-25T13:23:27.056 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.057 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.057 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.057 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-25T13:23:27.057 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.057 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.057 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.058 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-25T13:23:27.058 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.058 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.058 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.058 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-25T13:23:27.058 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.058 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.059 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.059 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-25T13:23:27.059 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.059 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.059 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.059 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-25T13:23:27.059 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.060 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.060 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.060 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-25T13:23:27.060 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.060 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.060 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.060 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-25T13:23:27.061 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.061 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.061 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.061 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-25T13:23:27.061 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.061 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.061 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.062 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-25T13:23:27.062 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.062 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.062 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.062 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-25T13:23:27.062 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-25T13:23:27.063 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.064 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.064 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.064 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-25T13:23:27.064 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.064 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.064 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.064 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-25T13:23:27.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.065 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.065 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-25T13:23:27.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.065 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.066 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-25T13:23:27.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.066 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.066 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-25T13:23:27.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.066 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-25T13:23:27.068 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-04-25T13:23:27.068 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 3, 2024-04-25T13:23:27.068 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 2, 2024-04-25T13:23:27.068 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 3, 2024-04-25T13:23:27.068 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-25T13:23:27.068 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-04-25T13:23:27.068 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-04-25T13:23:27.068 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 4215, 2024-04-25T13:23:27.069 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-25T13:23:27.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.069 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-25T13:23:27.069 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:27.069 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:27.069 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:27.069 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:27.069 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:27.069 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:27.070 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:27.070 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:27.070 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:27.070 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:27.070 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:27.070 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:27.070 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:27.070 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:27.070 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:27.071 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:27.071 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:27.071 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:27.071 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:27.071 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:27.071 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:27.071 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:27.071 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:27.071 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:27.072 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:27.072 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:27.072 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:27.072 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:27.072 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:27.072 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:27.072 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:27.072 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-25T13:23:27.072 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:27.073 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:27.073 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:27.073 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:27.073 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 190424, 2024-04-25T13:23:27.073 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 52, 2024-04-25T13:23:27.073 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:23:27.073 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-25T13:23:27.073 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-25T13:23:27.074 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-25T13:23:27.074 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:27.074 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:27.074 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:27.074 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:27.074 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8136, 2024-04-25T13:23:27.074 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 170, 2024-04-25T13:23:27.074 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:27.074 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:27.075 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:27.075 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-25T13:23:27.075 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 8512, 2024-04-25T13:23:27.075 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 13, 2024-04-25T13:23:27.075 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:27.075 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-25T13:23:27.075 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:27.075 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-25T13:23:27.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 2109, 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 2109, 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2109, 2024-04-25T13:23:27.076 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.952984066, 2024-04-25T13:23:27.077 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000451865 2024-04-25T13:23:27.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.077 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-25T13:23:27.077 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-25T13:23:27.077 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2109, 2024-04-25T13:23:27.077 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 2110 2024-04-25T13:23:27.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.077 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 2109, 2024-04-25T13:23:27.077 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 2109, 2024-04-25T13:23:27.078 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-04-25T13:23:27.078 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-25T13:23:27.078 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-25T13:23:27.078 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 1362, 2024-04-25T13:23:27.078 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-04-25T13:23:27.078 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 745, 2024-04-25T13:23:27.078 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-04-25T13:23:27.078 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-04-25T13:23:27.079 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-25T13:23:27.079 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-25T13:23:27.079 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-25T13:23:27.079 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-25T13:23:27.079 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-04-25T13:23:27.079 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-25T13:23:27.079 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-25T13:23:27.079 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-25T13:23:27.080 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2024-04-25T13:23:27.080 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-04-25T13:23:27.080 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:23:27.080 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-25T13:23:27.080 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:23:27.080 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-25T13:23:27.080 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-25T13:23:27.080 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-25T13:23:27.080 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:23:27.081 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-25T13:23:27.081 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:23:27.081 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-25T13:23:27.081 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-25T13:23:27.081 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-25T13:23:27.081 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-25T13:23:27.081 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-25T13:23:27.081 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-25T13:23:27.082 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-25T13:23:27.082 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-25T13:23:27.082 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-25T13:23:27.082 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-25T13:23:27.082 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-25T13:23:27.082 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-25T13:23:27.082 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-25T13:23:27.082 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-25T13:23:27.083 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-25T13:23:27.083 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-25T13:23:27.083 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-25T13:23:27.083 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 99, 2024-04-25T13:23:27.083 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-25T13:23:27.083 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 11, 2024-04-25T13:23:27.083 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 5, 2024-04-25T13:23:27.083 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 5, 2024-04-25T13:23:27.083 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-25T13:23:27.084 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-25T13:23:27.084 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-04-25T13:23:27.084 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-04-25T13:23:27.084 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-04-25T13:23:27.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.084 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-25T13:23:27.084 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-04-25T13:23:27.084 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:23:27.084 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-25T13:23:27.085 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-25T13:23:27.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.085 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-25T13:23:27.085 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-25T13:23:27.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.085 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-25T13:23:27.085 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-25T13:23:27.085 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:23:27.086 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-25T13:23:27.086 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-25T13:23:27.086 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-25T13:23:27.086 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-25T13:23:27.086 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-25T13:23:27.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.086 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:27.086 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:27.086 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T13:23:27.087 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:27.087 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 2345, 2024-04-25T13:23:27.087 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1989884, 2024-04-25T13:23:27.087 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:27.087 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 2345, 2024-04-25T13:23:27.087 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:27.087 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:27.087 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2345, 2024-04-25T13:23:27.087 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1989884, 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T13:23:27.088 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:27.089 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:27.089 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:27.089 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:27.089 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:27.089 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2109, 2024-04-25T13:23:27.089 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1965, 2024-04-25T13:23:27.089 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2109, 2024-04-25T13:23:27.089 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1965, 2024-04-25T13:23:27.089 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:27.090 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.090 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.090 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.090 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:27.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.090 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-25T13:23:27.090 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:27.090 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-25T13:23:27.090 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:27.091 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:27.091 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:27.091 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:27.091 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:27.091 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2109, 2024-04-25T13:23:27.091 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 2109, 2024-04-25T13:23:27.091 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2109, 2024-04-25T13:23:27.091 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2109, 2024-04-25T13:23:27.091 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:27.092 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.092 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.092 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:27.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.092 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:23:27.092 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:27.092 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T13:23:27.092 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:27.093 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:27.093 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:27.093 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:27.093 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:27.093 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:27.093 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:27.093 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-25T13:23:27.093 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-25T13:23:27.093 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:27.094 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.094 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.094 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:27.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:27.094 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55bdf4e4a6e0": { 2024-04-25T13:23:27.094 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:27.094 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T13:23:27.095 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:27.095 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:27.095 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:27.095 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:27.095 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:27.095 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:27.095 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:27.095 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-25T13:23:27.096 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-25T13:23:27.096 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:27.096 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:27.096 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:27.096 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:27.096 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:27.096 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:27.096 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T13:23:27.271 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:27 smithi164 ceph-mon[32418]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:23:27.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:27 smithi193 ceph-mon[32743]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:23:27.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:27 smithi072 ceph-mon[29199]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:23:27.681 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T13:23:27.681 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T13:23:27.681 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T13:23:27.681 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T13:23:27.682 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-25T13:23:27.682 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:23:27.682 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:23:27.682 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-25T13:23:27.682 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T13:23:27.682 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T13:23:27.922 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:23:29.251 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:29 smithi164 ceph-mon[32418]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:23:29.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:29 smithi193 ceph-mon[32743]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:23:29.519 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T13:23:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1530, 2024-04-25T13:23:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1532, 2024-04-25T13:23:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 548123, 2024-04-25T13:23:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 486550, 2024-04-25T13:23:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2024-04-25T13:23:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-25T13:23:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.142437546, 2024-04-25T13:23:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.055711612, 2024-04-25T13:23:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.032464115, 2024-04-25T13:23:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.040614325, 2024-04-25T13:23:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1530, 2024-04-25T13:23:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.055963758, 2024-04-25T13:23:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000036577 2024-04-25T13:23:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:23:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:23:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 6, 2024-04-25T13:23:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 160, 2024-04-25T13:23:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1976, 2024-04-25T13:23:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 439498, 2024-04-25T13:23:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 7, 2024-04-25T13:23:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-04-25T13:23:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.009492522, 2024-04-25T13:23:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.005760254, 2024-04-25T13:23:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.001942282, 2024-04-25T13:23:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000076521, 2024-04-25T13:23:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 157, 2024-04-25T13:23:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.013502719, 2024-04-25T13:23:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000086004 2024-04-25T13:23:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 438160 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 959, 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 945, 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1793389, 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 358639, 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 3, 2024-04-25T13:23:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-25T13:23:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.105969399, 2024-04-25T13:23:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.032378337, 2024-04-25T13:23:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.048552728, 2024-04-25T13:23:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.029027479, 2024-04-25T13:23:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 945, 2024-04-25T13:23:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.031047732, 2024-04-25T13:23:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000032854 2024-04-25T13:23:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1615952, 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 129360 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-25T13:23:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-25T13:23:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-25T13:23:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-25T13:23:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T13:23:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T13:23:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3774, 2024-04-25T13:23:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.096733612, 2024-04-25T13:23:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000025631 2024-04-25T13:23:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-25T13:23:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T13:23:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T13:23:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-25T13:23:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-04-25T13:23:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-04-25T13:23:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-25T13:23:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-25T13:23:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-25T13:23:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-25T13:23:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-25T13:23:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-04-25T13:23:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-25T13:23:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-25T13:23:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 12, 2024-04-25T13:23:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-04-25T13:23:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 12, 2024-04-25T13:23:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-25T13:23:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 12, 2024-04-25T13:23:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-25T13:23:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-04-25T13:23:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-04-25T13:23:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 2, 2024-04-25T13:23:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-04-25T13:23:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-04-25T13:23:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-25T13:23:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-25T13:23:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:23:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-25T13:23:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-25T13:23:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-25T13:23:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-25T13:23:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-25T13:23:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-25T13:23:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-25T13:23:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-25T13:23:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-25T13:23:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:23:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-25T13:23:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-04-25T13:23:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-04-25T13:23:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 6, 2024-04-25T13:23:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:23:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-25T13:23:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:23:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:23:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:23:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:23:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:23:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-25T13:23:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:23:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:23:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:23:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:23:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:23:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:23:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-25T13:23:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:23:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-25T13:23:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-25T13:23:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-25T13:23:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-25T13:23:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-25T13:23:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-25T13:23:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-04-25T13:23:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-25T13:23:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-25T13:23:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-25T13:23:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-25T13:23:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:23:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:23:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:23:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-25T13:23:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-25T13:23:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:23:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-25T13:23:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-25T13:23:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-25T13:23:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-25T13:23:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-04-25T13:23:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-25T13:23:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-25T13:23:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 103, 2024-04-25T13:23:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-25T13:23:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-25T13:23:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4286986, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 4286874, 2024-04-25T13:23:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-25T13:23:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 103 2024-04-25T13:23:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-25T13:23:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 13, 2024-04-25T13:23:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 13, 2024-04-25T13:23:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 0, 2024-04-25T13:23:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 8, 2024-04-25T13:23:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 8, 2024-04-25T13:23:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 0, 2024-04-25T13:23:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 12, 2024-04-25T13:23:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 12, 2024-04-25T13:23:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 0, 2024-04-25T13:23:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-04-25T13:23:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-04-25T13:23:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-04-25T13:23:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 51380, 2024-04-25T13:23:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215416 2024-04-25T13:23:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-25T13:23:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-04-25T13:23:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-25T13:23:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-25T13:23:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-25T13:23:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-25T13:23:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-25T13:23:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-25T13:23:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-25T13:23:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-25T13:23:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-25T13:23:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.551 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:29 smithi072 ceph-mon[29199]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:23:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-25T13:23:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-25T13:23:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-25T13:23:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-25T13:23:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-25T13:23:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-25T13:23:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-25T13:23:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-25T13:23:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-25T13:23:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-25T13:23:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-25T13:23:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-25T13:23:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-25T13:23:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-25T13:23:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-25T13:23:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-25T13:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-25T13:23:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 2, 2024-04-25T13:23:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 6, 2024-04-25T13:23:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 4, 2024-04-25T13:23:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 5, 2024-04-25T13:23:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-25T13:23:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-04-25T13:23:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-04-25T13:23:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 3618, 2024-04-25T13:23:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-25T13:23:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-25T13:23:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 189383, 2024-04-25T13:23:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 50, 2024-04-25T13:23:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:23:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-25T13:23:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-25T13:23:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-25T13:23:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8136, 2024-04-25T13:23:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 170, 2024-04-25T13:23:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-25T13:23:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 47746, 2024-04-25T13:23:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 220, 2024-04-25T13:23:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-25T13:23:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-25T13:23:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-25T13:23:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-25T13:23:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-25T13:23:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 2255, 2024-04-25T13:23:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 26, 2024-04-25T13:23:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 4, 2024-04-25T13:23:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 2255, 2024-04-25T13:23:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-25T13:23:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2255, 2024-04-25T13:23:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.863985575, 2024-04-25T13:23:29.572 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000383142 2024-04-25T13:23:29.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.572 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-25T13:23:29.572 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-25T13:23:29.572 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2255, 2024-04-25T13:23:29.572 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 2256 2024-04-25T13:23:29.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.572 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 2255, 2024-04-25T13:23:29.573 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 2255, 2024-04-25T13:23:29.573 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-04-25T13:23:29.573 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-25T13:23:29.573 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-25T13:23:29.573 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 1460, 2024-04-25T13:23:29.573 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-04-25T13:23:29.573 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 792, 2024-04-25T13:23:29.573 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-04-25T13:23:29.573 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-04-25T13:23:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-25T13:23:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-25T13:23:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-25T13:23:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-25T13:23:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-04-25T13:23:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-25T13:23:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-25T13:23:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-25T13:23:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 2, 2024-04-25T13:23:29.575 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-04-25T13:23:29.575 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:23:29.575 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-25T13:23:29.575 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:23:29.575 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-25T13:23:29.575 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-25T13:23:29.575 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-25T13:23:29.575 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:23:29.575 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-25T13:23:29.576 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:23:29.576 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-25T13:23:29.576 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-25T13:23:29.576 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-25T13:23:29.576 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-25T13:23:29.576 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-25T13:23:29.576 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-25T13:23:29.576 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-25T13:23:29.576 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-25T13:23:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 99, 2024-04-25T13:23:29.578 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-25T13:23:29.578 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 11, 2024-04-25T13:23:29.578 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 5, 2024-04-25T13:23:29.578 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 5, 2024-04-25T13:23:29.578 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-25T13:23:29.578 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-25T13:23:29.578 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-04-25T13:23:29.578 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-04-25T13:23:29.579 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-04-25T13:23:29.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.579 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-25T13:23:29.579 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-04-25T13:23:29.579 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:23:29.579 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-25T13:23:29.579 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-25T13:23:29.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.579 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-25T13:23:29.580 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-25T13:23:29.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.580 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-25T13:23:29.580 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-25T13:23:29.580 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:23:29.580 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-25T13:23:29.580 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-25T13:23:29.580 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-25T13:23:29.580 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-25T13:23:29.581 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-25T13:23:29.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.581 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:29.581 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:29.581 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T13:23:29.581 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:29.581 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 2491, 2024-04-25T13:23:29.581 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2104249, 2024-04-25T13:23:29.581 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:29.582 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 2491, 2024-04-25T13:23:29.582 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:29.582 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:29.582 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2491, 2024-04-25T13:23:29.582 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2104249, 2024-04-25T13:23:29.582 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:29.582 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.582 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:29.583 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:29.584 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:29.584 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2255, 2024-04-25T13:23:29.584 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 92692, 2024-04-25T13:23:29.584 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2255, 2024-04-25T13:23:29.584 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 92692, 2024-04-25T13:23:29.584 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:29.584 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.584 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.584 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.585 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:29.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.585 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-25T13:23:29.585 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:29.585 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-25T13:23:29.585 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:29.585 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:29.585 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:29.586 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:29.586 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:29.586 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2255, 2024-04-25T13:23:29.586 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 2255, 2024-04-25T13:23:29.586 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2255, 2024-04-25T13:23:29.586 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2255, 2024-04-25T13:23:29.586 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:29.586 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.587 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.587 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:29.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.587 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:23:29.587 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:29.587 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T13:23:29.587 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:29.587 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:29.588 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:29.588 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:29.588 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:29.588 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:29.588 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:29.588 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-25T13:23:29.588 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-25T13:23:29.588 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:29.588 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.589 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.589 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:29.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:29.589 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x5600bbad06e0": { 2024-04-25T13:23:29.589 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:29.589 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T13:23:29.589 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:29.589 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T13:23:29.590 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T13:23:29.590 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:29.590 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:29.590 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:29.590 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:29.590 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-25T13:23:29.590 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-25T13:23:29.590 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:29.591 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:29.591 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:29.591 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:29.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:29.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:29.591 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T13:23:30.249 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T13:23:30.249 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T13:23:30.249 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T13:23:30.249 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T13:23:30.249 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T13:23:30.249 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:23:30.250 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:23:30.250 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-25T13:23:30.250 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T13:23:30.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T13:23:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:23:31.307 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:31 smithi164 ceph-mon[32418]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:31.481 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:31 smithi193 ceph-mon[32743]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:31.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:31 smithi072 ceph-mon[29199]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:31.976 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T13:23:31.976 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:31.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 224, 2024-04-25T13:23:31.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 211, 2024-04-25T13:23:31.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1592416, 2024-04-25T13:23:31.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 128992, 2024-04-25T13:23:31.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2024-04-25T13:23:31.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 0, 2024-04-25T13:23:31.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.046729517, 2024-04-25T13:23:31.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.014099403, 2024-04-25T13:23:31.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.035973993, 2024-04-25T13:23:31.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.005150445, 2024-04-25T13:23:31.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:31.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 211, 2024-04-25T13:23:31.978 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.012125383, 2024-04-25T13:23:31.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000057466 2024-04-25T13:23:31.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:31.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:31.979 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:31.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:31.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1592416, 2024-04-25T13:23:31.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 128992 2024-04-25T13:23:31.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.979 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:31.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 2, 2024-04-25T13:23:31.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 160, 2024-04-25T13:23:31.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:23:31.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 184368, 2024-04-25T13:23:31.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-04-25T13:23:31.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:31.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.011405050, 2024-04-25T13:23:31.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.007375431, 2024-04-25T13:23:31.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.001721599, 2024-04-25T13:23:31.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:31.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:31.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 157, 2024-04-25T13:23:31.981 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.015375470, 2024-04-25T13:23:31.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000097932 2024-04-25T13:23:31.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.981 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:31.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:31.981 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:31.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:31.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.982 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:23:31.982 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 184368 2024-04-25T13:23:31.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.982 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:31.982 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 0, 2024-04-25T13:23:31.982 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 2, 2024-04-25T13:23:31.982 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:23:31.982 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 0, 2024-04-25T13:23:31.983 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 2, 2024-04-25T13:23:31.983 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 1, 2024-04-25T13:23:31.983 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.000381515, 2024-04-25T13:23:31.983 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:23:31.983 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.000320070, 2024-04-25T13:23:31.983 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:31.983 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:31.983 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:31.983 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:31.984 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:31.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.984 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:31.984 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:31.984 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:31.984 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:31.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.984 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:23:31.984 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:23:31.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.985 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-25T13:23:31.985 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 0, 2024-04-25T13:23:31.985 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-25T13:23:31.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.985 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-25T13:23:31.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:31.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:31.987 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:31.987 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:31.987 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:31.987 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:31.987 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:31.987 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:31.987 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:31.987 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:31.987 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:31.988 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:31.988 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:31.988 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:31.988 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:31.988 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:31.988 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:31.988 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:31.988 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-25T13:23:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 148994, 2024-04-25T13:23:31.990 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 31, 2024-04-25T13:23:31.990 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:23:31.990 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-25T13:23:31.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-25T13:23:31.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-25T13:23:31.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:31.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:31.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:31.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:31.991 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 0, 2024-04-25T13:23:31.991 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 0, 2024-04-25T13:23:31.991 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:31.991 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:31.991 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:31.991 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-25T13:23:31.991 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 928, 2024-04-25T13:23:31.991 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 2, 2024-04-25T13:23:31.991 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-25T13:23:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-25T13:23:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T13:23:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T13:23:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T13:23:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T13:23:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 222, 2024-04-25T13:23:31.993 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1558343, 2024-04-25T13:23:31.993 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:31.993 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 222, 2024-04-25T13:23:31.993 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T13:23:31.993 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T13:23:31.993 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 222, 2024-04-25T13:23:31.994 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1558343, 2024-04-25T13:23:31.994 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T13:23:31.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T13:23:31.994 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T13:23:31.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T13:23:31.994 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:31.994 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T13:23:31.995 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T13:23:32.567 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T13:23:32.567 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T13:23:32.567 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T13:23:32.567 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T13:23:32.567 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T13:23:32.567 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T13:23:32.568 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T13:23:32.568 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-25T13:23:32.568 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T13:23:32.568 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T13:23:32.819 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:23:33.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:33 smithi193 ceph-mon[32743]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:33.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:33 smithi072 ceph-mon[29199]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:33.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:33 smithi164 ceph-mon[32418]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:34.460 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T13:23:34.460 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:34.461 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 539, 2024-04-25T13:23:34.461 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 700, 2024-04-25T13:23:34.461 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1698823, 2024-04-25T13:23:34.461 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 339408, 2024-04-25T13:23:34.461 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2024-04-25T13:23:34.461 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:34.461 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.092176504, 2024-04-25T13:23:34.461 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.032933767, 2024-04-25T13:23:34.461 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.054566974, 2024-04-25T13:23:34.462 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.010078513, 2024-04-25T13:23:34.462 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:34.462 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 699, 2024-04-25T13:23:34.462 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.043706595, 2024-04-25T13:23:34.462 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000062527 2024-04-25T13:23:34.462 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.462 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:34.462 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 677, 2024-04-25T13:23:34.463 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000207552, 2024-04-25T13:23:34.463 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000306 2024-04-25T13:23:34.463 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.463 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1623632, 2024-04-25T13:23:34.463 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 134992 2024-04-25T13:23:34.463 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.463 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:34.463 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 26, 2024-04-25T13:23:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 181, 2024-04-25T13:23:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 8662, 2024-04-25T13:23:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 456217, 2024-04-25T13:23:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 9, 2024-04-25T13:23:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-04-25T13:23:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.012557572, 2024-04-25T13:23:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.006312338, 2024-04-25T13:23:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.003745078, 2024-04-25T13:23:34.465 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000587567, 2024-04-25T13:23:34.465 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:34.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 179, 2024-04-25T13:23:34.465 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.025262279, 2024-04-25T13:23:34.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000141130 2024-04-25T13:23:34.465 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.465 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:34.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8, 2024-04-25T13:23:34.465 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000001493, 2024-04-25T13:23:34.466 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000186 2024-04-25T13:23:34.466 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.466 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 4288, 2024-04-25T13:23:34.466 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 443168 2024-04-25T13:23:34.466 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.466 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:34.466 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 35, 2024-04-25T13:23:34.467 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 53, 2024-04-25T13:23:34.467 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 7905, 2024-04-25T13:23:34.467 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 61494, 2024-04-25T13:23:34.467 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 12, 2024-04-25T13:23:34.467 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:34.467 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.011949117, 2024-04-25T13:23:34.467 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.003498436, 2024-04-25T13:23:34.467 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.007379913, 2024-04-25T13:23:34.468 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000375605, 2024-04-25T13:23:34.468 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:34.468 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 51, 2024-04-25T13:23:34.468 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.010617030, 2024-04-25T13:23:34.468 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000208177 2024-04-25T13:23:34.468 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.468 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:34.468 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 55, 2024-04-25T13:23:34.468 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000011434, 2024-04-25T13:23:34.469 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000207 2024-04-25T13:23:34.469 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.469 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 2512, 2024-04-25T13:23:34.469 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 3952 2024-04-25T13:23:34.469 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.469 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-04-25T13:23:34.469 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-04-25T13:23:34.469 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-04-25T13:23:34.469 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.470 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-04-25T13:23:34.470 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-04-25T13:23:34.470 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-04-25T13:23:34.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 45, 2024-04-25T13:23:34.470 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002304780, 2024-04-25T13:23:34.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000051217 2024-04-25T13:23:34.470 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:34.470 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.470 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000008571, 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000004285 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 6, 2024-04-25T13:23:34.471 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 3, 2024-04-25T13:23:34.472 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-04-25T13:23:34.472 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3, 2024-04-25T13:23:34.472 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000505535, 2024-04-25T13:23:34.472 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000168511 2024-04-25T13:23:34.472 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.472 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-04-25T13:23:34.472 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 3, 2024-04-25T13:23:34.472 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-04-25T13:23:34.472 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-04-25T13:23:34.473 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 11, 2024-04-25T13:23:34.473 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-04-25T13:23:34.473 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-04-25T13:23:34.473 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 14, 2024-04-25T13:23:34.473 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 1, 2024-04-25T13:23:34.473 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 13, 2024-04-25T13:23:34.473 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-04-25T13:23:34.473 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 12, 2024-04-25T13:23:34.473 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-04-25T13:23:34.474 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-04-25T13:23:34.474 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-04-25T13:23:34.474 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 2, 2024-04-25T13:23:34.474 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 11, 2024-04-25T13:23:34.474 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 8, 2024-04-25T13:23:34.474 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 1, 2024-04-25T13:23:34.474 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 2, 2024-04-25T13:23:34.474 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:23:34.475 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-04-25T13:23:34.475 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-04-25T13:23:34.475 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-04-25T13:23:34.475 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-04-25T13:23:34.475 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-04-25T13:23:34.475 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-04-25T13:23:34.475 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-04-25T13:23:34.475 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-04-25T13:23:34.475 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 4, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 234, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:23:34.476 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:23:34.477 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:23:34.477 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:23:34.477 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-04-25T13:23:34.477 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:23:34.477 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:23:34.477 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:23:34.477 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:23:34.477 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:23:34.477 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:23:34.478 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-04-25T13:23:34.478 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:23:34.478 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-04-25T13:23:34.478 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-04-25T13:23:34.478 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.478 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-04-25T13:23:34.478 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-04-25T13:23:34.478 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 6, 2024-04-25T13:23:34.479 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 4, 2024-04-25T13:23:34.479 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 5, 2024-04-25T13:23:34.479 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-04-25T13:23:34.479 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-04-25T13:23:34.479 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-04-25T13:23:34.479 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:23:34.479 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-04-25T13:23:34.479 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-04-25T13:23:34.479 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:23:34.480 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-04-25T13:23:34.481 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-04-25T13:23:34.481 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:23:34.481 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-04-25T13:23:34.481 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-04-25T13:23:34.481 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.481 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-04-25T13:23:34.481 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-04-25T13:23:34.481 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 7, 2024-04-25T13:23:34.481 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-04-25T13:23:34.482 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-04-25T13:23:34.482 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 7, 2024-04-25T13:23:34.482 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-04-25T13:23:34.482 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-04-25T13:23:34.482 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 2, 2024-04-25T13:23:34.482 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-04-25T13:23:34.482 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-04-25T13:23:34.482 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-04-25T13:23:34.483 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-04-25T13:23:34.483 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-04-25T13:23:34.483 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-04-25T13:23:34.483 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-04-25T13:23:34.483 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4196264, 2024-04-25T13:23:34.483 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-04-25T13:23:34.483 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-04-25T13:23:34.483 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6, 2024-04-25T13:23:34.484 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.010722318, 2024-04-25T13:23:34.484 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001787053 2024-04-25T13:23:34.484 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.484 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-04-25T13:23:34.484 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.484 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-04-25T13:23:34.484 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 17, 2024-04-25T13:23:34.484 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 17, 2024-04-25T13:23:34.484 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 0, 2024-04-25T13:23:34.485 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 16, 2024-04-25T13:23:34.485 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 16, 2024-04-25T13:23:34.485 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 0, 2024-04-25T13:23:34.485 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 14, 2024-04-25T13:23:34.485 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 14, 2024-04-25T13:23:34.485 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 0, 2024-04-25T13:23:34.485 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-04-25T13:23:34.485 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 3, 2024-04-25T13:23:34.485 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 3, 2024-04-25T13:23:34.486 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 51304, 2024-04-25T13:23:34.486 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223608 2024-04-25T13:23:34.486 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.486 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-04-25T13:23:34.486 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 6, 2024-04-25T13:23:34.486 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-04-25T13:23:34.486 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 6, 2024-04-25T13:23:34.486 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 17, 2024-04-25T13:23:34.487 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-04-25T13:23:34.487 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-04-25T13:23:34.487 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.487 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.487 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.487 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.487 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-04-25T13:23:34.487 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-04-25T13:23:34.487 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000192247, 2024-04-25T13:23:34.488 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000192247 2024-04-25T13:23:34.488 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.488 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-04-25T13:23:34.488 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.488 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.488 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.488 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.488 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-04-25T13:23:34.488 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000313288, 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000156644 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-04-25T13:23:34.489 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-04-25T13:23:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-04-25T13:23:34.491 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.491 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.491 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.491 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.491 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-04-25T13:23:34.491 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.491 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.491 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.491 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.492 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-04-25T13:23:34.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.492 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.492 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.492 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-04-25T13:23:34.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.492 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.493 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.493 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.493 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-04-25T13:23:34.493 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.493 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.494 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.494 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.494 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-04-25T13:23:34.494 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.494 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.495 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.495 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.495 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-04-25T13:23:34.495 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.495 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.495 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.495 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.495 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-04-25T13:23:34.495 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-04-25T13:23:34.496 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-04-25T13:23:34.497 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.498 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.498 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.498 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.498 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-04-25T13:23:34.498 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.498 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.498 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.498 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.498 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-04-25T13:23:34.499 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.499 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.499 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.499 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.499 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-04-25T13:23:34.499 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.499 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.499 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.500 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.501 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.501 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-04-25T13:23:34.501 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.501 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.501 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.501 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.501 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-04-25T13:23:34.501 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.501 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.502 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.502 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.502 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-04-25T13:23:34.502 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.502 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.502 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.502 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.503 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.504 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:23:34.504 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:23:34.504 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-04-25T13:23:34.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.504 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.504 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:34.504 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.504 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 2, 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 2, 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 628, 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-04-25T13:23:34.505 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.506 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-04-25T13:23:34.506 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:34.506 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:34.506 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:34.506 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:34.506 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:34.506 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:34.507 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:34.507 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:34.507 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:34.507 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:34.507 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:34.507 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:34.507 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:34.507 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:34.507 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:34.508 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:34.508 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:34.508 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:34.508 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:34.508 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:34.508 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:34.508 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:34.508 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:34.508 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:34.509 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:34.509 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:34.509 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:34.509 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:34.509 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:34.509 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:34.509 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:34.509 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-04-25T13:23:34.510 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:34.510 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:34.510 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:34.510 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:34.510 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 247211, 2024-04-25T13:23:34.510 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 84, 2024-04-25T13:23:34.510 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 176, 2024-04-25T13:23:34.511 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 2, 2024-04-25T13:23:34.511 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-04-25T13:23:34.511 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-04-25T13:23:34.511 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:34.511 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:34.511 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:34.511 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:34.511 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8136, 2024-04-25T13:23:34.511 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 170, 2024-04-25T13:23:34.513 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:34.513 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:34.513 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:34.513 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-04-25T13:23:34.513 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 69128, 2024-04-25T13:23:34.514 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 264, 2024-04-25T13:23:34.514 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:34.514 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-04-25T13:23:34.514 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:34.514 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-04-25T13:23:34.514 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.514 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-04-25T13:23:34.514 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-04-25T13:23:34.515 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-04-25T13:23:34.515 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 44, 2024-04-25T13:23:34.515 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 12487, 2024-04-25T13:23:34.515 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-04-25T13:23:34.515 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 44, 2024-04-25T13:23:34.515 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-04-25T13:23:34.515 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 44, 2024-04-25T13:23:34.515 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.253995808, 2024-04-25T13:23:34.515 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005772632 2024-04-25T13:23:34.516 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.516 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-04-25T13:23:34.516 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-04-25T13:23:34.516 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 44, 2024-04-25T13:23:34.516 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 68 2024-04-25T13:23:34.516 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.516 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 44, 2024-04-25T13:23:34.516 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 0, 2024-04-25T13:23:34.516 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 44, 2024-04-25T13:23:34.517 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-04-25T13:23:34.517 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-04-25T13:23:34.517 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-04-25T13:23:34.517 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-04-25T13:23:34.517 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 0, 2024-04-25T13:23:34.517 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 6, 2024-04-25T13:23:34.517 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 7, 2024-04-25T13:23:34.517 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-04-25T13:23:34.517 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-04-25T13:23:34.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-04-25T13:23:34.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-04-25T13:23:34.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 6, 2024-04-25T13:23:34.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-04-25T13:23:34.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-04-25T13:23:34.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-04-25T13:23:34.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 0, 2024-04-25T13:23:34.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-04-25T13:23:34.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:23:34.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-04-25T13:23:34.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:23:34.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-04-25T13:23:34.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-04-25T13:23:34.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-04-25T13:23:34.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:23:34.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-04-25T13:23:34.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:23:34.520 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-04-25T13:23:34.520 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-04-25T13:23:34.520 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-04-25T13:23:34.520 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-04-25T13:23:34.520 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-04-25T13:23:34.520 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-04-25T13:23:34.520 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-04-25T13:23:34.520 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-04-25T13:23:34.520 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-04-25T13:23:34.521 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-04-25T13:23:34.521 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-04-25T13:23:34.521 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-04-25T13:23:34.521 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-04-25T13:23:34.521 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-04-25T13:23:34.521 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-04-25T13:23:34.521 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-04-25T13:23:34.521 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-04-25T13:23:34.521 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 99, 2024-04-25T13:23:34.522 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-04-25T13:23:34.522 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 11, 2024-04-25T13:23:34.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 8, 2024-04-25T13:23:34.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 8, 2024-04-25T13:23:34.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-04-25T13:23:34.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-04-25T13:23:34.522 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 17, 2024-04-25T13:23:34.522 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-04-25T13:23:34.523 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 2 2024-04-25T13:23:34.523 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.523 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-04-25T13:23:34.523 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-04-25T13:23:34.523 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:23:34.523 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-04-25T13:23:34.523 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-04-25T13:23:34.523 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.523 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-04-25T13:23:34.524 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-04-25T13:23:34.524 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.524 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-04-25T13:23:34.524 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-04-25T13:23:34.524 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:23:34.524 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-04-25T13:23:34.524 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-04-25T13:23:34.524 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-04-25T13:23:34.524 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-04-25T13:23:34.525 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-04-25T13:23:34.525 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.525 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:34.525 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:34.525 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-04-25T13:23:34.525 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:34.525 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 596, 2024-04-25T13:23:34.525 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1644530, 2024-04-25T13:23:34.525 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:34.526 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 596, 2024-04-25T13:23:34.526 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:34.526 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:34.526 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 596, 2024-04-25T13:23:34.526 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1644530, 2024-04-25T13:23:34.526 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:34.526 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.526 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.527 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.527 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:34.527 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.527 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-04-25T13:23:34.527 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:34.527 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-04-25T13:23:34.527 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:34.527 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:34.527 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:34.528 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:34.528 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:34.528 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 44, 2024-04-25T13:23:34.528 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 12487, 2024-04-25T13:23:34.528 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 44, 2024-04-25T13:23:34.528 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 12487, 2024-04-25T13:23:34.528 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:34.528 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.528 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.529 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.529 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:34.529 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.529 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-04-25T13:23:34.529 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:34.529 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-04-25T13:23:34.529 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:34.529 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:34.529 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 44, 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 44, 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 44, 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 44, 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.530 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.531 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:34.531 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.531 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:23:34.531 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:34.531 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-04-25T13:23:34.531 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:34.531 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:34.531 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:34.531 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:34.532 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:34.532 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:34.532 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:34.532 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-04-25T13:23:34.532 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-04-25T13:23:34.532 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:34.532 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.532 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.532 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.533 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:34.533 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:34.533 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x559d87076500": { 2024-04-25T13:23:34.533 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:34.533 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-04-25T13:23:34.533 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:34.533 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 7, 2024-04-25T13:23:34.533 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1960, 2024-04-25T13:23:34.533 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:34.534 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 7, 2024-04-25T13:23:34.534 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:34.534 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:34.534 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6, 2024-04-25T13:23:34.534 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1960, 2024-04-25T13:23:34.534 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:34.535 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:34.535 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:34.535 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:34.535 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:34.535 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:34.535 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T13:23:35.213 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:35 smithi193 ceph-mon[32743]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:35.240 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T13:23:35.241 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T13:23:35.241 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T13:23:35.241 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=6 2024-04-25T13:23:35.241 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-25T13:23:35.241 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:23:35.241 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:23:35.241 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-25T13:23:35.241 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T13:23:35.242 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T13:23:35.479 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:23:35.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:35 smithi072 ceph-mon[29199]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:35.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:35 smithi164 ceph-mon[32418]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:37.007 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T13:23:37.007 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:37.007 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 984, 2024-04-25T13:23:37.007 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 972, 2024-04-25T13:23:37.008 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1864896, 2024-04-25T13:23:37.008 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 368378, 2024-04-25T13:23:37.008 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-04-25T13:23:37.008 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 1, 2024-04-25T13:23:37.008 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.141171338, 2024-04-25T13:23:37.008 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.045671816, 2024-04-25T13:23:37.008 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.057870107, 2024-04-25T13:23:37.008 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.036455106, 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 971, 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.058314756, 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000060056 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.009 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.010 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1613904, 2024-04-25T13:23:37.010 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 130832 2024-04-25T13:23:37.010 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.010 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:37.010 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 8, 2024-04-25T13:23:37.010 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 163, 2024-04-25T13:23:37.010 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3916, 2024-04-25T13:23:37.010 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 443066, 2024-04-25T13:23:37.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-04-25T13:23:37.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-04-25T13:23:37.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.010878340, 2024-04-25T13:23:37.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.006111515, 2024-04-25T13:23:37.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.002787359, 2024-04-25T13:23:37.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000066006, 2024-04-25T13:23:37.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:37.011 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 161, 2024-04-25T13:23:37.011 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.014141569, 2024-04-25T13:23:37.012 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000087835 2024-04-25T13:23:37.012 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.012 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:37.012 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.012 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.012 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.012 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.012 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T13:23:37.012 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 442368 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1415, 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1417, 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 362730, 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 449977, 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 4, 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.097282157, 2024-04-25T13:23:37.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.032777978, 2024-04-25T13:23:37.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.023411631, 2024-04-25T13:23:37.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.034450598, 2024-04-25T13:23:37.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:37.014 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1415, 2024-04-25T13:23:37.014 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.046856474, 2024-04-25T13:23:37.014 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000033114 2024-04-25T13:23:37.014 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-04-25T13:23:37.015 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-04-25T13:23:37.016 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.016 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-04-25T13:23:37.016 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-04-25T13:23:37.016 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-04-25T13:23:37.016 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3662, 2024-04-25T13:23:37.016 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.103678945, 2024-04-25T13:23:37.016 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000028312 2024-04-25T13:23:37.016 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:37.016 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.017 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-04-25T13:23:37.017 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-04-25T13:23:37.017 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-04-25T13:23:37.017 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.017 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.017 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.017 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:37.017 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.018 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-04-25T13:23:37.018 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-04-25T13:23:37.018 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-04-25T13:23:37.018 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-04-25T13:23:37.018 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.018 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.018 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.018 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.019 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-04-25T13:23:37.019 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-04-25T13:23:37.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-04-25T13:23:37.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-04-25T13:23:37.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-04-25T13:23:37.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-04-25T13:23:37.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-04-25T13:23:37.019 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 0, 2024-04-25T13:23:37.020 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-04-25T13:23:37.020 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 0, 2024-04-25T13:23:37.020 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-04-25T13:23:37.020 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 0, 2024-04-25T13:23:37.020 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-04-25T13:23:37.020 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-04-25T13:23:37.020 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-04-25T13:23:37.020 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 1, 2024-04-25T13:23:37.020 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-04-25T13:23:37.021 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-04-25T13:23:37.021 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-04-25T13:23:37.021 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-04-25T13:23:37.021 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:23:37.021 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-04-25T13:23:37.021 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-04-25T13:23:37.021 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-04-25T13:23:37.021 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-04-25T13:23:37.021 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-04-25T13:23:37.022 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-04-25T13:23:37.022 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-04-25T13:23:37.022 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-04-25T13:23:37.022 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-04-25T13:23:37.022 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:23:37.022 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-04-25T13:23:37.022 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-04-25T13:23:37.022 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-04-25T13:23:37.022 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-04-25T13:23:37.023 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:23:37.023 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-04-25T13:23:37.023 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:23:37.023 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:23:37.023 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:23:37.023 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:23:37.023 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:23:37.023 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-04-25T13:23:37.023 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-04-25T13:23:37.024 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.025 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-04-25T13:23:37.025 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-04-25T13:23:37.025 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-04-25T13:23:37.025 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-04-25T13:23:37.025 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-04-25T13:23:37.025 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-04-25T13:23:37.025 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-04-25T13:23:37.025 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-04-25T13:23:37.026 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:23:37.026 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-04-25T13:23:37.026 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-04-25T13:23:37.026 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-04-25T13:23:37.026 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-04-25T13:23:37.026 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-04-25T13:23:37.026 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:23:37.026 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:23:37.026 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-04-25T13:23:37.027 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-04-25T13:23:37.027 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:23:37.027 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:23:37.027 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:23:37.027 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-04-25T13:23:37.027 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-04-25T13:23:37.027 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:23:37.027 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-04-25T13:23:37.027 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-04-25T13:23:37.028 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.028 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-04-25T13:23:37.028 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-04-25T13:23:37.028 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-04-25T13:23:37.028 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-04-25T13:23:37.028 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-04-25T13:23:37.028 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 7, 2024-04-25T13:23:37.028 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-04-25T13:23:37.028 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4196347, 2024-04-25T13:23:37.029 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 4196206, 2024-04-25T13:23:37.030 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-04-25T13:23:37.030 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.030 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.030 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.030 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.030 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 7 2024-04-25T13:23:37.030 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.030 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-04-25T13:23:37.030 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 3, 2024-04-25T13:23:37.031 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 3, 2024-04-25T13:23:37.031 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 0, 2024-04-25T13:23:37.031 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 1, 2024-04-25T13:23:37.031 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 1, 2024-04-25T13:23:37.031 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 0, 2024-04-25T13:23:37.031 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 0, 2024-04-25T13:23:37.031 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 0, 2024-04-25T13:23:37.031 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 0, 2024-04-25T13:23:37.031 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-04-25T13:23:37.032 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-04-25T13:23:37.032 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-04-25T13:23:37.032 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 49924, 2024-04-25T13:23:37.032 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 215416 2024-04-25T13:23:37.032 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.032 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-04-25T13:23:37.032 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-04-25T13:23:37.032 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-04-25T13:23:37.032 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-04-25T13:23:37.033 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-04-25T13:23:37.033 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-04-25T13:23:37.033 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-04-25T13:23:37.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.033 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.033 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.033 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-04-25T13:23:37.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.034 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.034 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.034 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-04-25T13:23:37.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.034 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.034 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.034 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.035 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-04-25T13:23:37.036 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.036 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.036 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.036 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.036 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-04-25T13:23:37.036 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.036 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.036 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.036 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.037 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-04-25T13:23:37.037 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.037 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.037 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.037 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.037 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-04-25T13:23:37.037 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.037 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.037 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.038 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.038 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-04-25T13:23:37.038 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.038 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.038 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.038 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.038 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-04-25T13:23:37.038 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.038 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.039 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.039 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.039 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-04-25T13:23:37.039 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.039 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.040 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.040 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.040 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-04-25T13:23:37.040 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.040 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.040 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.040 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.040 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-04-25T13:23:37.041 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.041 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.041 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.041 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.041 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-04-25T13:23:37.041 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.041 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.041 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.041 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.042 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-04-25T13:23:37.042 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.042 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.042 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.042 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.042 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-04-25T13:23:37.042 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.042 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.042 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.043 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.043 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-04-25T13:23:37.043 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.043 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.043 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.043 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.043 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-04-25T13:23:37.043 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.043 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.044 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.044 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.044 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-04-25T13:23:37.044 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.044 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.044 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.044 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.044 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-04-25T13:23:37.044 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.045 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.045 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.045 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.045 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-04-25T13:23:37.045 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.045 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.045 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.045 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.045 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-04-25T13:23:37.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.046 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.046 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.046 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-04-25T13:23:37.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.046 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.046 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.047 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.048 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-04-25T13:23:37.048 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.048 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.048 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.048 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.048 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-04-25T13:23:37.048 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.048 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.048 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.049 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.049 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-04-25T13:23:37.049 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.049 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.049 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.049 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.049 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:23:37.049 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:23:37.049 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-04-25T13:23:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-04-25T13:23:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-04-25T13:23:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-04-25T13:23:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 2, 2024-04-25T13:23:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-04-25T13:23:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-04-25T13:23:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-04-25T13:23:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-04-25T13:23:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 3141, 2024-04-25T13:23:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-04-25T13:23:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-04-25T13:23:37.052 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:37.052 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:37.052 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:37.052 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:37.052 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:37.052 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:37.052 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:37.052 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:37.052 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:37.053 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:37.054 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:37.054 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:37.054 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:37.054 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:37.054 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:37.054 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:37.054 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:37.054 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:37.055 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 181343, 2024-04-25T13:23:37.056 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 47, 2024-04-25T13:23:37.056 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:23:37.056 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-04-25T13:23:37.056 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-04-25T13:23:37.056 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-04-25T13:23:37.056 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:37.056 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:37.056 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:37.056 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8136, 2024-04-25T13:23:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 170, 2024-04-25T13:23:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-04-25T13:23:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 8512, 2024-04-25T13:23:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 13, 2024-04-25T13:23:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:37.058 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-04-25T13:23:37.058 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:37.058 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-04-25T13:23:37.058 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.058 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-04-25T13:23:37.058 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-04-25T13:23:37.058 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-04-25T13:23:37.058 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 2164, 2024-04-25T13:23:37.058 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-04-25T13:23:37.059 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-04-25T13:23:37.059 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 2164, 2024-04-25T13:23:37.059 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-04-25T13:23:37.059 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2164, 2024-04-25T13:23:37.059 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.922984928, 2024-04-25T13:23:37.059 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000426517 2024-04-25T13:23:37.059 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.059 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-04-25T13:23:37.059 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-04-25T13:23:37.060 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2164, 2024-04-25T13:23:37.060 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2165 2024-04-25T13:23:37.060 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.060 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 2164, 2024-04-25T13:23:37.060 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 2164, 2024-04-25T13:23:37.060 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-04-25T13:23:37.060 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-04-25T13:23:37.060 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-04-25T13:23:37.060 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 1406, 2024-04-25T13:23:37.061 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-04-25T13:23:37.061 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 756, 2024-04-25T13:23:37.061 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-04-25T13:23:37.061 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-04-25T13:23:37.061 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-04-25T13:23:37.061 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-04-25T13:23:37.061 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-04-25T13:23:37.061 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-04-25T13:23:37.061 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-04-25T13:23:37.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-04-25T13:23:37.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-04-25T13:23:37.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-04-25T13:23:37.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-04-25T13:23:37.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-04-25T13:23:37.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:23:37.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-04-25T13:23:37.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:23:37.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-04-25T13:23:37.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-04-25T13:23:37.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-04-25T13:23:37.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:23:37.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-04-25T13:23:37.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:23:37.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-04-25T13:23:37.063 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-04-25T13:23:37.063 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-04-25T13:23:37.063 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-04-25T13:23:37.064 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-04-25T13:23:37.064 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-04-25T13:23:37.064 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-04-25T13:23:37.064 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-04-25T13:23:37.064 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-04-25T13:23:37.064 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-04-25T13:23:37.064 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-04-25T13:23:37.064 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-04-25T13:23:37.064 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-04-25T13:23:37.065 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-04-25T13:23:37.065 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-04-25T13:23:37.065 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-04-25T13:23:37.065 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-04-25T13:23:37.065 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 99, 2024-04-25T13:23:37.065 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-04-25T13:23:37.065 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 11, 2024-04-25T13:23:37.065 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 4, 2024-04-25T13:23:37.065 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 4, 2024-04-25T13:23:37.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-04-25T13:23:37.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-04-25T13:23:37.066 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-04-25T13:23:37.066 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-04-25T13:23:37.066 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-04-25T13:23:37.066 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.066 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-04-25T13:23:37.066 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-04-25T13:23:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:23:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-04-25T13:23:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-04-25T13:23:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-04-25T13:23:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-04-25T13:23:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-04-25T13:23:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-04-25T13:23:37.069 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:37.069 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 2403, 2024-04-25T13:23:37.069 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 2000061, 2024-04-25T13:23:37.069 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:37.069 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 2403, 2024-04-25T13:23:37.069 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:37.069 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:37.069 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2403, 2024-04-25T13:23:37.069 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2000061, 2024-04-25T13:23:37.070 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:37.070 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.070 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.070 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.070 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:37.070 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.070 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-04-25T13:23:37.070 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:37.070 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-04-25T13:23:37.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:37.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:37.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:37.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:37.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:37.071 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2164, 2024-04-25T13:23:37.071 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2048, 2024-04-25T13:23:37.071 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2164, 2024-04-25T13:23:37.071 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2048, 2024-04-25T13:23:37.072 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:37.072 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.072 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.072 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.072 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:37.072 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.072 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-04-25T13:23:37.072 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:37.072 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-04-25T13:23:37.073 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:37.073 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:37.073 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:37.073 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:37.073 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:37.073 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2164, 2024-04-25T13:23:37.073 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2164, 2024-04-25T13:23:37.073 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2164, 2024-04-25T13:23:37.073 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2164, 2024-04-25T13:23:37.074 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:37.074 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.074 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.074 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.074 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:37.074 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.074 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:23:37.074 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-04-25T13:23:37.075 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-04-25T13:23:37.076 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:37.076 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.076 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.076 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.076 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:37.076 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:37.076 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55af462546e0": { 2024-04-25T13:23:37.076 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:37.076 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-04-25T13:23:37.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:37.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:37.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:37.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:37.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:37.077 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:37.077 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:37.077 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-04-25T13:23:37.077 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-04-25T13:23:37.078 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:37.078 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:37.078 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:37.078 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:37.078 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:37.078 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:37.078 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T13:23:37.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:37 smithi193 ceph-mon[32743]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:37.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:37 smithi072 ceph-mon[29199]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:37.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:37 smithi164 ceph-mon[32418]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:37.805 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T13:23:37.805 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T13:23:37.805 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T13:23:37.805 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T13:23:37.805 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T13:23:37.805 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:23:37.806 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:23:37.806 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-25T13:23:37.806 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T13:23:37.806 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T13:23:38.044 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:23:39.359 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:39 smithi193 ceph-mon[32743]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:39.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:39 smithi072 ceph-mon[29199]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:39.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:39 smithi164 ceph-mon[32418]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:39.622 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T13:23:39.622 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:39.622 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 39, 2024-04-25T13:23:39.623 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 171, 2024-04-25T13:23:39.623 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 11809, 2024-04-25T13:23:39.623 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 356147, 2024-04-25T13:23:39.623 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 12, 2024-04-25T13:23:39.623 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-04-25T13:23:39.623 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.013631819, 2024-04-25T13:23:39.623 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.006494225, 2024-04-25T13:23:39.623 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.003718498, 2024-04-25T13:23:39.624 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000628798, 2024-04-25T13:23:39.624 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:39.624 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 166, 2024-04-25T13:23:39.624 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.028848121, 2024-04-25T13:23:39.624 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000173783 2024-04-25T13:23:39.624 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.624 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:39.624 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 13, 2024-04-25T13:23:39.625 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000001786, 2024-04-25T13:23:39.625 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000137 2024-04-25T13:23:39.625 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.625 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 4784, 2024-04-25T13:23:39.625 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 319088 2024-04-25T13:23:39.625 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.625 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:39.625 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 547, 2024-04-25T13:23:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 705, 2024-04-25T13:23:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1705783, 2024-04-25T13:23:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 331000, 2024-04-25T13:23:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-04-25T13:23:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-04-25T13:23:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.092154294, 2024-04-25T13:23:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.034049489, 2024-04-25T13:23:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.057008438, 2024-04-25T13:23:39.627 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.010483785, 2024-04-25T13:23:39.627 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:39.627 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 708, 2024-04-25T13:23:39.627 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.045900079, 2024-04-25T13:23:39.627 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000064830 2024-04-25T13:23:39.627 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.627 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:39.627 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 698, 2024-04-25T13:23:39.628 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000211507, 2024-04-25T13:23:39.629 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000303 2024-04-25T13:23:39.629 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.629 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1627872, 2024-04-25T13:23:39.629 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 135632 2024-04-25T13:23:39.629 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.629 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:39.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 34, 2024-04-25T13:23:39.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 93, 2024-04-25T13:23:39.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 8569, 2024-04-25T13:23:39.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 175581, 2024-04-25T13:23:39.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 11, 2024-04-25T13:23:39.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-04-25T13:23:39.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.009409585, 2024-04-25T13:23:39.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.002892522, 2024-04-25T13:23:39.631 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.005409401, 2024-04-25T13:23:39.631 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000386082, 2024-04-25T13:23:39.631 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:39.631 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 90, 2024-04-25T13:23:39.631 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.015816948, 2024-04-25T13:23:39.631 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000175743 2024-04-25T13:23:39.631 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.631 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:39.631 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 68, 2024-04-25T13:23:39.632 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000011016, 2024-04-25T13:23:39.632 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000162 2024-04-25T13:23:39.632 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.632 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 5472, 2024-04-25T13:23:39.632 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 141888 2024-04-25T13:23:39.632 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.632 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-04-25T13:23:39.632 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-04-25T13:23:39.633 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-04-25T13:23:39.633 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.633 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-04-25T13:23:39.633 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-04-25T13:23:39.633 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-04-25T13:23:39.633 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 53, 2024-04-25T13:23:39.633 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002416540, 2024-04-25T13:23:39.634 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000045595 2024-04-25T13:23:39.634 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:39.634 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.634 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-04-25T13:23:39.634 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-04-25T13:23:39.634 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-04-25T13:23:39.634 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-04-25T13:23:39.635 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000007378, 2024-04-25T13:23:39.635 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000003689 2024-04-25T13:23:39.635 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:39.635 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.635 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-04-25T13:23:39.635 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 11, 2024-04-25T13:23:39.635 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 7, 2024-04-25T13:23:39.635 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-04-25T13:23:39.636 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 7, 2024-04-25T13:23:39.636 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001710816, 2024-04-25T13:23:39.636 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000244402 2024-04-25T13:23:39.636 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.636 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-04-25T13:23:39.636 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 4, 2024-04-25T13:23:39.636 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-04-25T13:23:39.636 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-04-25T13:23:39.637 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 11, 2024-04-25T13:23:39.637 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-04-25T13:23:39.637 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-04-25T13:23:39.637 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 14, 2024-04-25T13:23:39.637 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 1, 2024-04-25T13:23:39.637 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 13, 2024-04-25T13:23:39.637 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-04-25T13:23:39.637 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 12, 2024-04-25T13:23:39.638 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 1, 2024-04-25T13:23:39.638 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 1, 2024-04-25T13:23:39.638 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 1, 2024-04-25T13:23:39.638 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 2, 2024-04-25T13:23:39.638 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 23, 2024-04-25T13:23:39.638 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 14, 2024-04-25T13:23:39.638 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 1, 2024-04-25T13:23:39.639 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 4, 2024-04-25T13:23:39.639 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:23:39.639 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-04-25T13:23:39.639 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-04-25T13:23:39.639 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-04-25T13:23:39.639 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-04-25T13:23:39.639 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-04-25T13:23:39.639 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-04-25T13:23:39.640 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-04-25T13:23:39.640 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-04-25T13:23:39.640 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-04-25T13:23:39.640 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:23:39.640 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-04-25T13:23:39.640 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 4, 2024-04-25T13:23:39.640 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 234, 2024-04-25T13:23:39.641 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-04-25T13:23:39.641 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:23:39.641 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-04-25T13:23:39.641 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:23:39.641 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:23:39.641 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:23:39.641 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:23:39.641 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:23:39.641 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-04-25T13:23:39.642 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:23:39.642 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:23:39.642 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:23:39.642 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:23:39.642 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:23:39.642 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:23:39.642 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-04-25T13:23:39.642 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:23:39.643 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-04-25T13:23:39.643 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 1 2024-04-25T13:23:39.643 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.643 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-04-25T13:23:39.643 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-04-25T13:23:39.643 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 6, 2024-04-25T13:23:39.643 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 4, 2024-04-25T13:23:39.643 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 7, 2024-04-25T13:23:39.644 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-04-25T13:23:39.644 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-04-25T13:23:39.644 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-04-25T13:23:39.644 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:23:39.644 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-04-25T13:23:39.644 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-04-25T13:23:39.644 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-04-25T13:23:39.644 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-04-25T13:23:39.644 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-04-25T13:23:39.645 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:23:39.645 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:23:39.645 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-04-25T13:23:39.645 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-04-25T13:23:39.645 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:23:39.645 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:23:39.645 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:23:39.645 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-04-25T13:23:39.646 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-04-25T13:23:39.646 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:23:39.646 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-04-25T13:23:39.646 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-04-25T13:23:39.646 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.646 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-04-25T13:23:39.646 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-04-25T13:23:39.646 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 7, 2024-04-25T13:23:39.647 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-04-25T13:23:39.647 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-04-25T13:23:39.647 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 7, 2024-04-25T13:23:39.647 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-04-25T13:23:39.647 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-04-25T13:23:39.647 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 2, 2024-04-25T13:23:39.647 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-04-25T13:23:39.647 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-04-25T13:23:39.647 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-04-25T13:23:39.648 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-04-25T13:23:39.648 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-04-25T13:23:39.648 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-04-25T13:23:39.648 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-04-25T13:23:39.648 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4196347, 2024-04-25T13:23:39.648 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-04-25T13:23:39.648 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-04-25T13:23:39.648 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6, 2024-04-25T13:23:39.649 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.013273017, 2024-04-25T13:23:39.649 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002212169 2024-04-25T13:23:39.649 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.649 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-04-25T13:23:39.649 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.649 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-04-25T13:23:39.649 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 17, 2024-04-25T13:23:39.649 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 17, 2024-04-25T13:23:39.650 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 0, 2024-04-25T13:23:39.650 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 16, 2024-04-25T13:23:39.650 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 17, 2024-04-25T13:23:39.650 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 1, 2024-04-25T13:23:39.650 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 14, 2024-04-25T13:23:39.650 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 15, 2024-04-25T13:23:39.650 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 1, 2024-04-25T13:23:39.651 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 1, 2024-04-25T13:23:39.651 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 4, 2024-04-25T13:23:39.651 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 3, 2024-04-25T13:23:39.651 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 53136, 2024-04-25T13:23:39.651 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223608 2024-04-25T13:23:39.651 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.651 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-04-25T13:23:39.651 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 13, 2024-04-25T13:23:39.651 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-04-25T13:23:39.652 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 11, 2024-04-25T13:23:39.652 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 16, 2024-04-25T13:23:39.652 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-04-25T13:23:39.652 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-04-25T13:23:39.652 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.652 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.652 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.652 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.653 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-04-25T13:23:39.653 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5, 2024-04-25T13:23:39.653 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001164358, 2024-04-25T13:23:39.653 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000232871 2024-04-25T13:23:39.653 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.653 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-04-25T13:23:39.653 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.653 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.653 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.654 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.654 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-04-25T13:23:39.654 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.654 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.654 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.654 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.654 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-04-25T13:23:39.654 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-04-25T13:23:39.654 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000546458, 2024-04-25T13:23:39.655 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000273229 2024-04-25T13:23:39.655 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.655 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-04-25T13:23:39.655 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.655 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.655 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.655 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.655 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-04-25T13:23:39.655 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.656 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.656 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.656 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.656 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-04-25T13:23:39.656 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.656 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.656 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.656 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.656 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.657 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-04-25T13:23:39.658 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.658 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.658 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.658 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.658 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-04-25T13:23:39.658 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.659 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.659 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.659 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.660 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-04-25T13:23:39.660 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.660 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.660 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.660 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.660 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-04-25T13:23:39.660 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.660 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.661 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.661 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.661 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-04-25T13:23:39.661 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.661 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.661 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.661 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.661 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-04-25T13:23:39.661 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.662 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.662 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.662 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.662 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-04-25T13:23:39.662 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.662 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.662 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.662 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.662 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-04-25T13:23:39.663 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.663 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.663 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.663 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.663 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-04-25T13:23:39.663 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.663 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.663 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.663 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.664 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-04-25T13:23:39.664 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.664 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.664 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.664 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.664 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-04-25T13:23:39.664 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.664 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.665 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.665 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.665 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-04-25T13:23:39.665 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.665 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.665 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.665 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.666 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-04-25T13:23:39.666 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.666 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.666 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.666 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.666 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-04-25T13:23:39.666 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.666 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.667 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.667 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.667 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-04-25T13:23:39.667 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.667 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.667 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.667 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.667 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-04-25T13:23:39.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.668 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.668 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.668 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-04-25T13:23:39.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.668 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.669 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.669 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-04-25T13:23:39.669 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.669 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.669 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.669 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.669 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-04-25T13:23:39.669 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.669 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.670 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.670 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.670 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:23:39.670 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:23:39.670 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-04-25T13:23:39.670 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.670 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.670 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.671 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:39.671 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.671 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-04-25T13:23:39.671 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-04-25T13:23:39.671 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-04-25T13:23:39.671 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 2, 2024-04-25T13:23:39.671 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-04-25T13:23:39.671 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-04-25T13:23:39.672 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 4, 2024-04-25T13:23:39.672 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 2, 2024-04-25T13:23:39.672 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 575, 2024-04-25T13:23:39.672 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-04-25T13:23:39.672 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.672 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-04-25T13:23:39.672 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:39.673 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:39.673 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:39.673 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:39.673 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:39.673 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:39.673 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:39.673 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:39.673 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:39.673 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:39.674 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:39.674 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:39.674 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:39.674 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:39.674 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:39.674 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:39.674 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:39.674 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:39.674 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:39.675 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:39.675 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:39.675 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:39.675 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:39.675 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:39.675 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:39.675 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-04-25T13:23:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:39.677 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:39.677 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 248423, 2024-04-25T13:23:39.677 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 90, 2024-04-25T13:23:39.677 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 88, 2024-04-25T13:23:39.677 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 1, 2024-04-25T13:23:39.677 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-04-25T13:23:39.677 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-04-25T13:23:39.677 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:39.677 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:39.678 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:39.678 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:39.678 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8136, 2024-04-25T13:23:39.678 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 170, 2024-04-25T13:23:39.678 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:39.678 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:39.678 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:39.679 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-04-25T13:23:39.679 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 69456, 2024-04-25T13:23:39.679 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 261, 2024-04-25T13:23:39.679 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:39.679 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-04-25T13:23:39.679 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:39.679 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-04-25T13:23:39.679 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.679 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-04-25T13:23:39.680 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-04-25T13:23:39.680 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-04-25T13:23:39.680 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 46, 2024-04-25T13:23:39.680 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 12670, 2024-04-25T13:23:39.680 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-04-25T13:23:39.680 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 46, 2024-04-25T13:23:39.680 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-04-25T13:23:39.680 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 46, 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.271995524, 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005912946 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 46, 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 70 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 46, 2024-04-25T13:23:39.681 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 0, 2024-04-25T13:23:39.682 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 46, 2024-04-25T13:23:39.682 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-04-25T13:23:39.682 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-04-25T13:23:39.682 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-04-25T13:23:39.682 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-04-25T13:23:39.682 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 0, 2024-04-25T13:23:39.682 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 6, 2024-04-25T13:23:39.682 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 9, 2024-04-25T13:23:39.683 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-04-25T13:23:39.683 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-04-25T13:23:39.683 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-04-25T13:23:39.683 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-04-25T13:23:39.683 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 6, 2024-04-25T13:23:39.683 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-04-25T13:23:39.683 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-04-25T13:23:39.683 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-04-25T13:23:39.683 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 0, 2024-04-25T13:23:39.684 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-04-25T13:23:39.684 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:23:39.684 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-04-25T13:23:39.684 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:23:39.684 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-04-25T13:23:39.684 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-04-25T13:23:39.684 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-04-25T13:23:39.684 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:23:39.684 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-04-25T13:23:39.685 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:23:39.685 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-04-25T13:23:39.685 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-04-25T13:23:39.685 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-04-25T13:23:39.685 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-04-25T13:23:39.685 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-04-25T13:23:39.685 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-04-25T13:23:39.685 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-04-25T13:23:39.685 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-04-25T13:23:39.686 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-04-25T13:23:39.686 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-04-25T13:23:39.686 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-04-25T13:23:39.686 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-04-25T13:23:39.686 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-04-25T13:23:39.686 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-04-25T13:23:39.686 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-04-25T13:23:39.686 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-04-25T13:23:39.686 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-04-25T13:23:39.687 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 99, 2024-04-25T13:23:39.687 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-04-25T13:23:39.687 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 11, 2024-04-25T13:23:39.687 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 9, 2024-04-25T13:23:39.687 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 9, 2024-04-25T13:23:39.687 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-04-25T13:23:39.687 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-04-25T13:23:39.687 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 17, 2024-04-25T13:23:39.687 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-04-25T13:23:39.688 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 2 2024-04-25T13:23:39.688 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.688 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-04-25T13:23:39.688 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-04-25T13:23:39.688 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:23:39.688 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-04-25T13:23:39.688 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-04-25T13:23:39.688 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.688 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-04-25T13:23:39.689 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-04-25T13:23:39.689 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.689 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-04-25T13:23:39.689 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-04-25T13:23:39.689 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:23:39.689 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-04-25T13:23:39.689 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-04-25T13:23:39.689 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-04-25T13:23:39.689 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-04-25T13:23:39.690 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-04-25T13:23:39.690 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.690 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:39.690 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:39.690 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-04-25T13:23:39.690 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:39.690 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 616, 2024-04-25T13:23:39.690 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1652746, 2024-04-25T13:23:39.691 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:39.691 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 616, 2024-04-25T13:23:39.691 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:39.691 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:39.691 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 616, 2024-04-25T13:23:39.691 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1652746, 2024-04-25T13:23:39.691 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:39.691 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:39.692 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:39.693 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:39.693 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:39.693 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 46, 2024-04-25T13:23:39.693 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 12670, 2024-04-25T13:23:39.693 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 46, 2024-04-25T13:23:39.693 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 12670, 2024-04-25T13:23:39.693 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:39.693 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.693 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.694 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.694 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:39.694 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.694 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-04-25T13:23:39.694 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:39.694 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-04-25T13:23:39.694 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:39.694 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:39.695 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:39.695 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:39.695 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:39.695 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 46, 2024-04-25T13:23:39.695 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 46, 2024-04-25T13:23:39.695 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 46, 2024-04-25T13:23:39.695 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 46, 2024-04-25T13:23:39.695 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:39.696 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.696 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.696 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.696 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:39.696 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.696 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:23:39.696 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:39.696 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-04-25T13:23:39.696 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:39.697 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-04-25T13:23:39.697 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-04-25T13:23:39.697 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:39.697 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-04-25T13:23:39.697 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:39.697 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:39.697 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-04-25T13:23:39.697 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-04-25T13:23:39.697 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:39.698 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.698 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.698 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.698 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:39.698 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:39.698 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55aeec892500": { 2024-04-25T13:23:39.698 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:39.698 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-04-25T13:23:39.699 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:39.699 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 7, 2024-04-25T13:23:39.699 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 2043, 2024-04-25T13:23:39.699 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:39.699 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 7, 2024-04-25T13:23:39.699 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:39.699 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:39.699 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6, 2024-04-25T13:23:39.700 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2043, 2024-04-25T13:23:39.700 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:39.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:39.700 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:39.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:39.700 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:39.700 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:39.700 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T13:23:40.328 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T13:23:40.328 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T13:23:40.329 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T13:23:40.329 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=6 2024-04-25T13:23:40.329 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-25T13:23:40.329 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:23:40.329 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:23:40.329 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-25T13:23:40.329 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T13:23:40.330 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T13:23:40.564 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:23:41.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:41 smithi193 ceph-mon[32743]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:41.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:41 smithi072 ceph-mon[29199]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:41.557 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:41 smithi164 ceph-mon[32418]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:42.268 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T13:23:42.268 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:23:42.268 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 4, 2024-04-25T13:23:42.269 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 4, 2024-04-25T13:23:42.269 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3040, 2024-04-25T13:23:42.269 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 448, 2024-04-25T13:23:42.269 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 4, 2024-04-25T13:23:42.269 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 0, 2024-04-25T13:23:42.269 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.001256040, 2024-04-25T13:23:42.269 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.000034076, 2024-04-25T13:23:42.269 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.000490517, 2024-04-25T13:23:42.269 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:42.270 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:42.270 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3, 2024-04-25T13:23:42.270 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000701414, 2024-04-25T13:23:42.270 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000233804 2024-04-25T13:23:42.270 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.270 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:42.270 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:42.271 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:42.271 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:42.271 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.271 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T13:23:42.271 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:23:42.271 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.271 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:23:42.271 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 225, 2024-04-25T13:23:42.271 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 214, 2024-04-25T13:23:42.272 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1590800, 2024-04-25T13:23:42.272 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 131264, 2024-04-25T13:23:42.272 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 2, 2024-04-25T13:23:42.272 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 1, 2024-04-25T13:23:42.272 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.048324691, 2024-04-25T13:23:42.272 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.013530787, 2024-04-25T13:23:42.272 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.039023368, 2024-04-25T13:23:42.272 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.005651599, 2024-04-25T13:23:42.272 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:42.273 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 213, 2024-04-25T13:23:42.273 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.010230812, 2024-04-25T13:23:42.273 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000048031 2024-04-25T13:23:42.273 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.273 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:42.273 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:42.273 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:42.273 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:42.274 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.274 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1590800, 2024-04-25T13:23:42.274 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 131264 2024-04-25T13:23:42.274 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.274 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:23:42.274 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 2, 2024-04-25T13:23:42.274 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 163, 2024-04-25T13:23:42.274 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:23:42.275 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 184896, 2024-04-25T13:23:42.275 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-04-25T13:23:42.275 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-04-25T13:23:42.275 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.012036148, 2024-04-25T13:23:42.275 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.008414177, 2024-04-25T13:23:42.275 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.001141277, 2024-04-25T13:23:42.275 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:23:42.275 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:23:42.275 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 160, 2024-04-25T13:23:42.276 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.019758674, 2024-04-25T13:23:42.276 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000123491 2024-04-25T13:23:42.276 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.276 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:23:42.276 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:42.276 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:42.276 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:42.276 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.277 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:23:42.277 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 184896 2024-04-25T13:23:42.277 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.277 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-04-25T13:23:42.277 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 0, 2024-04-25T13:23:42.277 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-04-25T13:23:42.277 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.277 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-04-25T13:23:42.278 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:23:42.278 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-04-25T13:23:42.278 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:23:42.278 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:23:42.278 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:23:42.278 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:23:42.278 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:23:42.278 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:23:42.279 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:23:42.279 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:23:42.279 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:23:42.279 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:23:42.279 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:23:42.279 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:23:42.279 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:23:42.279 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-04-25T13:23:42.279 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:23:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-04-25T13:23:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:23:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:23:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:23:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:23:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:23:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:23:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:23:42.281 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-04-25T13:23:42.281 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:23:42.281 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:23:42.281 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:23:42.281 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-04-25T13:23:42.281 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-04-25T13:23:42.281 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-04-25T13:23:42.281 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:23:42.281 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:23:42.282 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:23:42.282 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:23:42.282 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 148923, 2024-04-25T13:23:42.282 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 29, 2024-04-25T13:23:42.282 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:23:42.282 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-04-25T13:23:42.282 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-04-25T13:23:42.285 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-04-25T13:23:42.285 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:23:42.285 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-04-25T13:23:42.286 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:23:42.286 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-04-25T13:23:42.286 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 0, 2024-04-25T13:23:42.286 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 0, 2024-04-25T13:23:42.286 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:23:42.286 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:23:42.286 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-04-25T13:23:42.287 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-04-25T13:23:42.287 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 928, 2024-04-25T13:23:42.287 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 2, 2024-04-25T13:23:42.287 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-04-25T13:23:42.287 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-04-25T13:23:42.287 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-04-25T13:23:42.287 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-04-25T13:23:42.287 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T13:23:42.288 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:23:42.288 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-04-25T13:23:42.288 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-04-25T13:23:42.288 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-04-25T13:23:42.288 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 227, 2024-04-25T13:23:42.288 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1559057, 2024-04-25T13:23:42.288 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-04-25T13:23:42.288 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 227, 2024-04-25T13:23:42.288 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-04-25T13:23:42.289 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-04-25T13:23:42.289 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 227, 2024-04-25T13:23:42.289 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1559057, 2024-04-25T13:23:42.289 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-04-25T13:23:42.289 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-04-25T13:23:42.289 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-04-25T13:23:42.289 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-04-25T13:23:42.289 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:42.289 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T13:23:42.290 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T13:23:42.942 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T13:23:42.943 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T13:23:42.943 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-25T13:23:42.943 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-25T13:23:42.943 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T13:23:42.943 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T13:23:42.943 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T13:23:42.943 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-25T13:23:42.944 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.dir_split', 'mds.imported'} 2024-04-25T13:23:42.954 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T13:23:42.966 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T13:23:42.966 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:23:42.966 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:23:42.992 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:23:42.992 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:23:43.050 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:23:43.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.200+0000 7fe272c79640 1 -- 172.21.15.72:0/4238093079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c107260 msgr2=0x7fe26c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.200+0000 7fe272c79640 1 --2- 172.21.15.72:0/4238093079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c107260 0x7fe26c10ee30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe264009ca0 tx=0x7fe26402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.200+0000 7fe272c79640 1 -- 172.21.15.72:0/4238093079 shutdown_connections 2024-04-25T13:23:43.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.200+0000 7fe272c79640 1 --2- 172.21.15.72:0/4238093079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c107260 0x7fe26c10ee30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.200+0000 7fe272c79640 1 --2- 172.21.15.72:0/4238093079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26c102b40 0x7fe26c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.200+0000 7fe272c79640 1 --2- 172.21.15.72:0/4238093079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe26c102190 0x7fe26c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.200+0000 7fe272c79640 1 -- 172.21.15.72:0/4238093079 >> 172.21.15.72:0/4238093079 conn(0x7fe26c079310 msgr2=0x7fe26c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:43.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.200+0000 7fe272c79640 1 -- 172.21.15.72:0/4238093079 shutdown_connections 2024-04-25T13:23:43.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.200+0000 7fe272c79640 1 -- 172.21.15.72:0/4238093079 wait complete. 2024-04-25T13:23:43.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe272c79640 1 Processor -- start 2024-04-25T13:23:43.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe272c79640 1 -- start start 2024-04-25T13:23:43.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe272c79640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26c102190 0x7fe26c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe272c79640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe26c107260 0x7fe26c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe272c79640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c19cfd0 0x7fe26c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe272c79640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1121d0 con 0x7fe26c19cfd0 2024-04-25T13:23:43.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe272c79640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c112050 con 0x7fe26c107260 2024-04-25T13:23:43.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe272c79640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c112350 con 0x7fe26c102190 2024-04-25T13:23:43.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe271216640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c19cfd0 0x7fe26c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe271216640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c19cfd0 0x7fe26c19d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54862/0 (socket says 172.21.15.72:54862) 2024-04-25T13:23:43.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe271216640 1 -- 172.21.15.72:0/918938583 learned_addr learned my addr 172.21.15.72:0/918938583 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:43.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe270a15640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26c102190 0x7fe26c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe26bfff640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe26c107260 0x7fe26c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe271216640 1 -- 172.21.15.72:0/918938583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26c102190 msgr2=0x7fe26c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe271216640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26c102190 0x7fe26c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe271216640 1 -- 172.21.15.72:0/918938583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe26c107260 msgr2=0x7fe26c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe271216640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe26c107260 0x7fe26c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.201+0000 7fe271216640 1 -- 172.21.15.72:0/918938583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c19dac0 con 0x7fe26c19cfd0 2024-04-25T13:23:43.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.202+0000 7fe271216640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c19cfd0 0x7fe26c19d480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe264005cc0 tx=0x7fe26402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:43.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.202+0000 7fe269ffb640 1 -- 172.21.15.72:0/918938583 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264003bf0 con 0x7fe26c19cfd0 2024-04-25T13:23:43.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.202+0000 7fe269ffb640 1 -- 172.21.15.72:0/918938583 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe264003d90 con 0x7fe26c19cfd0 2024-04-25T13:23:43.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.202+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c19dd50 con 0x7fe26c19cfd0 2024-04-25T13:23:43.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.202+0000 7fe269ffb640 1 -- 172.21.15.72:0/918938583 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26402fbc0 con 0x7fe26c19cfd0 2024-04-25T13:23:43.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.202+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c1a55a0 con 0x7fe26c19cfd0 2024-04-25T13:23:43.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.203+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c0691c0 con 0x7fe26c19cfd0 2024-04-25T13:23:43.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.204+0000 7fe269ffb640 1 -- 172.21.15.72:0/918938583 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe26402fd60 con 0x7fe26c19cfd0 2024-04-25T13:23:43.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.205+0000 7fe269ffb640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe25c0a3da0 0x7fe25c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.205+0000 7fe270a15640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe25c0a3da0 0x7fe25c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.205+0000 7fe269ffb640 1 -- 172.21.15.72:0/918938583 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fe2640f1e70 con 0x7fe26c19cfd0 2024-04-25T13:23:43.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.205+0000 7fe270a15640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe25c0a3da0 0x7fe25c0a6260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe260005fd0 tx=0x7fe260005850 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:43.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.209+0000 7fe269ffb640 1 -- 172.21.15.72:0/918938583 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2640be600 con 0x7fe26c19cfd0 2024-04-25T13:23:43.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.387+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe26c10a4f0 con 0x7fe26c19cfd0 2024-04-25T13:23:43.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.389+0000 7fe269ffb640 1 -- 172.21.15.72:0/918938583 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23208 (secure 0 0 0) 0x7fe2640bdf00 con 0x7fe26c19cfd0 2024-04-25T13:23:43.390 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:23:43.390 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":99,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:23:17.416503+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/3348457169":"2024-04-26T13:13:03.464543+0000","172.21.15.72:0/4004940606":"2024-04-26T13:13:03.464543+0000","172.21.15.72:0/3501290813":"2024-04-26T13:13:03.464543+0000","172.21.15.72:6800/3348457169":"2024-04-26T13:13:03.464543+0000","172.21.15.72:0/1633858068":"2024-04-26T13:13:03.464543+0000","172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2743588764":"2024-04-26T13:13:03.464543+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:23:43.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe25c0a3da0 msgr2=0x7fe25c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe25c0a3da0 0x7fe25c0a6260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe260005fd0 tx=0x7fe260005850 comp rx=0 tx=0).stop 2024-04-25T13:23:43.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c19cfd0 msgr2=0x7fe26c19d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c19cfd0 0x7fe26c19d480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe264005cc0 tx=0x7fe26402f380 comp rx=0 tx=0).stop 2024-04-25T13:23:43.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 shutdown_connections 2024-04-25T13:23:43.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe25c0a3da0 0x7fe25c0a6260 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe26c19cfd0 0x7fe26c19d480 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe26c107260 0x7fe26c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 --2- 172.21.15.72:0/918938583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26c102190 0x7fe26c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 >> 172.21.15.72:0/918938583 conn(0x7fe26c079310 msgr2=0x7fe26c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:43.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 shutdown_connections 2024-04-25T13:23:43.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.391+0000 7fe272c79640 1 -- 172.21.15.72:0/918938583 wait complete. 2024-04-25T13:23:43.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:06:33.150709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:11:02.352089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:11:03.565033+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:23:43.404 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:23:43.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.505+0000 7f19e88d0640 1 -- 172.21.15.72:0/2433593295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e41011a0 msgr2=0x7f19e4101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.505+0000 7f19e88d0640 1 --2- 172.21.15.72:0/2433593295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e41011a0 0x7f19e4101600 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009930 tx=0x7f19d802a420 comp rx=0 tx=0).stop 2024-04-25T13:23:43.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.505+0000 7f19e88d0640 1 -- 172.21.15.72:0/2433593295 shutdown_connections 2024-04-25T13:23:43.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.505+0000 7f19e88d0640 1 --2- 172.21.15.72:0/2433593295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e4101b40 0x7f19e410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.505+0000 7f19e88d0640 1 --2- 172.21.15.72:0/2433593295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e41011a0 0x7f19e4101600 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.505+0000 7f19e88d0640 1 --2- 172.21.15.72:0/2433593295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19e4107030 0x7f19e4107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.505+0000 7f19e88d0640 1 -- 172.21.15.72:0/2433593295 >> 172.21.15.72:0/2433593295 conn(0x7f19e4076c50 msgr2=0x7f19e4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:43.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.505+0000 7f19e88d0640 1 -- 172.21.15.72:0/2433593295 shutdown_connections 2024-04-25T13:23:43.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.505+0000 7f19e88d0640 1 -- 172.21.15.72:0/2433593295 wait complete. 2024-04-25T13:23:43.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e88d0640 1 Processor -- start 2024-04-25T13:23:43.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e88d0640 1 -- start start 2024-04-25T13:23:43.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e88d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e41011a0 0x7f19e4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e88d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19e4101b40 0x7f19e4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e88d0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e4107030 0x7f19e4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e88d0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f19e41110e0 con 0x7f19e4107030 2024-04-25T13:23:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e88d0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4110f60 con 0x7f19e41011a0 2024-04-25T13:23:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e88d0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4111260 con 0x7f19e4101b40 2024-04-25T13:23:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e2d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e4107030 0x7f19e4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e2d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e4107030 0x7f19e4197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54872/0 (socket says 172.21.15.72:54872) 2024-04-25T13:23:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e2d9d640 1 -- 172.21.15.72:0/390139870 learned_addr learned my addr 172.21.15.72:0/390139870 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e1d9b640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19e4101b40 0x7f19e4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e2d9d640 1 -- 172.21.15.72:0/390139870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19e4101b40 msgr2=0x7f19e4197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e2d9d640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19e4101b40 0x7f19e4197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e259c640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e41011a0 0x7f19e4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e2d9d640 1 -- 172.21.15.72:0/390139870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e41011a0 msgr2=0x7f19e4197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e2d9d640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e41011a0 0x7f19e4197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.506+0000 7f19e2d9d640 1 -- 172.21.15.72:0/390139870 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e41a6c80 con 0x7f19e4107030 2024-04-25T13:23:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.507+0000 7f19e2d9d640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e4107030 0x7f19e4197e90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009fd0 tx=0x7f19dc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.507+0000 7f19d37fe640 1 -- 172.21.15.72:0/390139870 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc017070 con 0x7f19e4107030 2024-04-25T13:23:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.507+0000 7f19d37fe640 1 -- 172.21.15.72:0/390139870 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19dc003e20 con 0x7f19e4107030 2024-04-25T13:23:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.507+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e41a6f70 con 0x7f19e4107030 2024-04-25T13:23:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.507+0000 7f19d37fe640 1 -- 172.21.15.72:0/390139870 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc010460 con 0x7f19e4107030 2024-04-25T13:23:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.507+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41a7440 con 0x7f19e4107030 2024-04-25T13:23:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.508+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e4102590 con 0x7f19e4107030 2024-04-25T13:23:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.509+0000 7f19d37fe640 1 -- 172.21.15.72:0/390139870 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f19dc003970 con 0x7f19e4107030 2024-04-25T13:23:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.509+0000 7f19d37fe640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f19cc0a3d20 0x7f19cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.510+0000 7f19e259c640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f19cc0a3d20 0x7f19cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.510+0000 7f19d37fe640 1 -- 172.21.15.72:0/390139870 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f19dc0d05e0 con 0x7f19e4107030 2024-04-25T13:23:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.510+0000 7f19e259c640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f19cc0a3d20 0x7f19cc0a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f19d4005d80 tx=0x7f19d4005c90 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:43.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.514+0000 7f19d37fe640 1 -- 172.21.15.72:0/390139870 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19dc09cc60 con 0x7f19e4107030 2024-04-25T13:23:43.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.692+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f19e406ab40 con 0x7f19e4107030 2024-04-25T13:23:43.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.692+0000 7f19d37fe640 1 -- 172.21.15.72:0/390139870 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f19e406ab40 con 0x7f19e4107030 2024-04-25T13:23:43.693 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-25T13:23:43.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f19cc0a3d20 msgr2=0x7f19cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f19cc0a3d20 0x7f19cc0a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f19d4005d80 tx=0x7f19d4005c90 comp rx=0 tx=0).stop 2024-04-25T13:23:43.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e4107030 msgr2=0x7f19e4197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e4107030 0x7f19e4197e90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009fd0 tx=0x7f19dc00b260 comp rx=0 tx=0).stop 2024-04-25T13:23:43.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 shutdown_connections 2024-04-25T13:23:43.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f19cc0a3d20 0x7f19cc0a61e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19e4107030 0x7f19e4197e90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19e4101b40 0x7f19e4197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 --2- 172.21.15.72:0/390139870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e41011a0 0x7f19e4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 >> 172.21.15.72:0/390139870 conn(0x7f19e4076c50 msgr2=0x7f19e41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:43.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 shutdown_connections 2024-04-25T13:23:43.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.694+0000 7f19e88d0640 1 -- 172.21.15.72:0/390139870 wait complete. 2024-04-25T13:23:43.704 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:43 smithi072 ceph-mon[29199]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:43.705 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:43 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/918938583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:43.706 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:23:43.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:43 smithi193 ceph-mon[32743]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:43.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:43 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/918938583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:43.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.806+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1001987826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f381010b0 msgr2=0x7f2f38101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.806+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1001987826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f381010b0 0x7f2f38101490 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009900 tx=0x7f2f2802a410 comp rx=0 tx=0).stop 2024-04-25T13:23:43.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.806+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1001987826 shutdown_connections 2024-04-25T13:23:43.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.806+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1001987826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f38106180 0x7f2f3810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.806+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1001987826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f38101a60 0x7f2f38105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.806+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1001987826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f381010b0 0x7f2f38101490 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.806+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1001987826 >> 172.21.15.72:0/1001987826 conn(0x7f2f380771d0 msgr2=0x7f2f380775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:43.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.806+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1001987826 shutdown_connections 2024-04-25T13:23:43.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.806+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1001987826 wait complete. 2024-04-25T13:23:43.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f3d80b640 1 Processor -- start 2024-04-25T13:23:43.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f3d80b640 1 -- start start 2024-04-25T13:23:43.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f3d80b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f381010b0 0x7f2f38199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f3d80b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f38106180 0x7f2f381999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f3d80b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f38199f50 0x7f2f381a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f3d80b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2f381110c0 con 0x7f2f38199f50 2024-04-25T13:23:43.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f3d80b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f38110f40 con 0x7f2f381010b0 2024-04-25T13:23:43.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f3d80b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f38111240 con 0x7f2f38106180 2024-04-25T13:23:43.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f377fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f38199f50 0x7f2f381a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f377fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f38199f50 0x7f2f381a4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54900/0 (socket says 172.21.15.72:54900) 2024-04-25T13:23:43.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f377fe640 1 -- 172.21.15.72:0/1789744936 learned_addr learned my addr 172.21.15.72:0/1789744936 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:43.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f36ffd640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f381010b0 0x7f2f38199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f377fe640 1 -- 172.21.15.72:0/1789744936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f38106180 msgr2=0x7f2f381999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f377fe640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f38106180 0x7f2f381999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f377fe640 1 -- 172.21.15.72:0/1789744936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f381010b0 msgr2=0x7f2f38199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f377fe640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f381010b0 0x7f2f38199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f377fe640 1 -- 172.21.15.72:0/1789744936 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f381a4cd0 con 0x7f2f38199f50 2024-04-25T13:23:43.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.807+0000 7f2f36ffd640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f381010b0 0x7f2f38199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:23:43.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.808+0000 7f2f377fe640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f38199f50 0x7f2f381a4500 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000cce0 tx=0x7f2f3000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:43.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.808+0000 7f2f3c809640 1 -- 172.21.15.72:0/1789744936 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3000f450 con 0x7f2f38199f50 2024-04-25T13:23:43.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.808+0000 7f2f3c809640 1 -- 172.21.15.72:0/1789744936 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f3000faa0 con 0x7f2f38199f50 2024-04-25T13:23:43.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.808+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f381a4fc0 con 0x7f2f38199f50 2024-04-25T13:23:43.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.808+0000 7f2f3c809640 1 -- 172.21.15.72:0/1789744936 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30018b70 con 0x7f2f38199f50 2024-04-25T13:23:43.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.808+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f381a5480 con 0x7f2f38199f50 2024-04-25T13:23:43.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:43 smithi164 ceph-mon[32418]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:43.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:43 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/918938583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:43.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.809+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f380691c0 con 0x7f2f38199f50 2024-04-25T13:23:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.810+0000 7f2f3c809640 1 -- 172.21.15.72:0/1789744936 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f30018d10 con 0x7f2f38199f50 2024-04-25T13:23:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.810+0000 7f2f3c809640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2f240a3d20 0x7f2f240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.811+0000 7f2f36ffd640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2f240a3d20 0x7f2f240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.811+0000 7f2f3c809640 1 -- 172.21.15.72:0/1789744936 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f2f300d0c90 con 0x7f2f38199f50 2024-04-25T13:23:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.811+0000 7f2f36ffd640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2f240a3d20 0x7f2f240a61e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f28031440 tx=0x7f2f280095f0 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.814+0000 7f2f3c809640 1 -- 172.21.15.72:0/1789744936 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f30014030 con 0x7f2f38199f50 2024-04-25T13:23:43.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.989+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f38101a60 con 0x7f2f38199f50 2024-04-25T13:23:43.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.989+0000 7f2f3c809640 1 -- 172.21.15.72:0/1789744936 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f2f3009d3a0 con 0x7f2f38199f50 2024-04-25T13:23:43.990 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-25T13:23:43.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2f240a3d20 msgr2=0x7f2f240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2f240a3d20 0x7f2f240a61e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f28031440 tx=0x7f2f280095f0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f38199f50 msgr2=0x7f2f381a4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f38199f50 0x7f2f381a4500 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000cce0 tx=0x7f2f3000a620 comp rx=0 tx=0).stop 2024-04-25T13:23:43.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 shutdown_connections 2024-04-25T13:23:43.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f2f240a3d20 0x7f2f240a61e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f38199f50 0x7f2f381a4500 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f38106180 0x7f2f381999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 --2- 172.21.15.72:0/1789744936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f381010b0 0x7f2f38199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 >> 172.21.15.72:0/1789744936 conn(0x7f2f380771d0 msgr2=0x7f2f380febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:43.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 shutdown_connections 2024-04-25T13:23:43.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:43.991+0000 7f2f3d80b640 1 -- 172.21.15.72:0/1789744936 wait complete. 2024-04-25T13:23:44.003 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:23:44.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.104+0000 7f9e19274640 1 -- 172.21.15.72:0/1079603450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14102190 msgr2=0x7f9e14102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.104+0000 7f9e19274640 1 --2- 172.21.15.72:0/1079603450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14102190 0x7f9e14102570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009900 tx=0x7f9e0402a410 comp rx=0 tx=0).stop 2024-04-25T13:23:44.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.104+0000 7f9e19274640 1 -- 172.21.15.72:0/1079603450 shutdown_connections 2024-04-25T13:23:44.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.104+0000 7f9e19274640 1 --2- 172.21.15.72:0/1079603450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e14107260 0x7f9e1410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.104+0000 7f9e19274640 1 --2- 172.21.15.72:0/1079603450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e14102b40 0x7f9e14106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.104+0000 7f9e19274640 1 --2- 172.21.15.72:0/1079603450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14102190 0x7f9e14102570 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.104+0000 7f9e19274640 1 -- 172.21.15.72:0/1079603450 >> 172.21.15.72:0/1079603450 conn(0x7f9e140793a0 msgr2=0x7f9e140797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:44.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.104+0000 7f9e19274640 1 -- 172.21.15.72:0/1079603450 shutdown_connections 2024-04-25T13:23:44.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.104+0000 7f9e19274640 1 -- 172.21.15.72:0/1079603450 wait complete. 2024-04-25T13:23:44.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e19274640 1 Processor -- start 2024-04-25T13:23:44.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e19274640 1 -- start start 2024-04-25T13:23:44.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e19274640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e14102190 0x7f9e141a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:44.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e19274640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14107260 0x7f9e141a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:44.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e19274640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1419a8c0 0x7f9e1419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:44.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e19274640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9e14112240 con 0x7f9e14107260 2024-04-25T13:23:44.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e19274640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e141120c0 con 0x7f9e14102190 2024-04-25T13:23:44.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e19274640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e141123c0 con 0x7f9e1419a8c0 2024-04-25T13:23:44.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e0bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14107260 0x7f9e141a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:44.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e0bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14107260 0x7f9e141a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54920/0 (socket says 172.21.15.72:54920) 2024-04-25T13:23:44.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e0bfff640 1 -- 172.21.15.72:0/1355433431 learned_addr learned my addr 172.21.15.72:0/1355433431 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:44.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e137fe640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1419a8c0 0x7f9e1419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:44.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e12ffd640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e14102190 0x7f9e141a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:44.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e0bfff640 1 -- 172.21.15.72:0/1355433431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1419a8c0 msgr2=0x7f9e1419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e0bfff640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1419a8c0 0x7f9e1419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e0bfff640 1 -- 172.21.15.72:0/1355433431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e14102190 msgr2=0x7f9e141a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e0bfff640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e14102190 0x7f9e141a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e0bfff640 1 -- 172.21.15.72:0/1355433431 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1419b3e0 con 0x7f9e14107260 2024-04-25T13:23:44.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e12ffd640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e14102190 0x7f9e141a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:23:44.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.105+0000 7f9e0bfff640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14107260 0x7f9e141a0c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00d740 tx=0x7f9dfc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:44.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.106+0000 7f9e10ff9640 1 -- 172.21.15.72:0/1355433431 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc017070 con 0x7f9e14107260 2024-04-25T13:23:44.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.106+0000 7f9e10ff9640 1 -- 172.21.15.72:0/1355433431 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dfc00de50 con 0x7f9e14107260 2024-04-25T13:23:44.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.106+0000 7f9e10ff9640 1 -- 172.21.15.72:0/1355433431 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc004ce0 con 0x7f9e14107260 2024-04-25T13:23:44.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.106+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1419b6d0 con 0x7f9e14107260 2024-04-25T13:23:44.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.106+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e141a78d0 con 0x7f9e14107260 2024-04-25T13:23:44.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.107+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e140691c0 con 0x7f9e14107260 2024-04-25T13:23:44.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.108+0000 7f9e10ff9640 1 -- 172.21.15.72:0/1355433431 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9dfc004e80 con 0x7f9e14107260 2024-04-25T13:23:44.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.108+0000 7f9e10ff9640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e000a3d20 0x7f9e000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:44.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.108+0000 7f9e12ffd640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e000a3d20 0x7f9e000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:44.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.108+0000 7f9e10ff9640 1 -- 172.21.15.72:0/1355433431 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f9dfc0d06a0 con 0x7f9e14107260 2024-04-25T13:23:44.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.109+0000 7f9e12ffd640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e000a3d20 0x7f9e000a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000c00 tx=0x7f9e04031400 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:44.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.112+0000 7f9e10ff9640 1 -- 172.21.15.72:0/1355433431 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dfc09cd20 con 0x7f9e14107260 2024-04-25T13:23:44.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.290+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e14102bd0 con 0x7f9e14107260 2024-04-25T13:23:44.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.291+0000 7f9e10ff9640 1 -- 172.21.15.72:0/1355433431 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f9dfc09c620 con 0x7f9e14107260 2024-04-25T13:23:44.292 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-25T13:23:44.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e000a3d20 msgr2=0x7f9e000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e000a3d20 0x7f9e000a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000c00 tx=0x7f9e04031400 comp rx=0 tx=0).stop 2024-04-25T13:23:44.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14107260 msgr2=0x7f9e141a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14107260 0x7f9e141a0c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00d740 tx=0x7f9dfc00dc00 comp rx=0 tx=0).stop 2024-04-25T13:23:44.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 shutdown_connections 2024-04-25T13:23:44.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f9e000a3d20 0x7f9e000a61e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1419a8c0 0x7f9e1419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e14107260 0x7f9e141a0c50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 --2- 172.21.15.72:0/1355433431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e14102190 0x7f9e141a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 >> 172.21.15.72:0/1355433431 conn(0x7f9e140793a0 msgr2=0x7f9e14100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:44.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 shutdown_connections 2024-04-25T13:23:44.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.293+0000 7f9e19274640 1 -- 172.21.15.72:0/1355433431 wait complete. 2024-04-25T13:23:44.305 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:23:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.404+0000 7faa108f6640 1 -- 172.21.15.72:0/4239273383 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 msgr2=0x7faa0c10ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.404+0000 7faa108f6640 1 --2- 172.21.15.72:0/4239273383 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 0x7faa0c10ee10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faa04009ca0 tx=0x7faa0402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.404+0000 7faa108f6640 1 -- 172.21.15.72:0/4239273383 shutdown_connections 2024-04-25T13:23:44.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.404+0000 7faa108f6640 1 --2- 172.21.15.72:0/4239273383 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 0x7faa0c10ee10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.404+0000 7faa108f6640 1 --2- 172.21.15.72:0/4239273383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c069d00 0x7faa0c106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.404+0000 7faa108f6640 1 --2- 172.21.15.72:0/4239273383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa0c069350 0x7faa0c069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.404+0000 7faa108f6640 1 -- 172.21.15.72:0/4239273383 >> 172.21.15.72:0/4239273383 conn(0x7faa0c077cb0 msgr2=0x7faa0c0780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:44.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.404+0000 7faa108f6640 1 -- 172.21.15.72:0/4239273383 shutdown_connections 2024-04-25T13:23:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.404+0000 7faa108f6640 1 -- 172.21.15.72:0/4239273383 wait complete. 2024-04-25T13:23:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa108f6640 1 Processor -- start 2024-04-25T13:23:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa108f6640 1 -- start start 2024-04-25T13:23:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa108f6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa0c069350 0x7faa0c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa108f6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 0x7faa0c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa108f6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c19cf70 0x7faa0c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa108f6640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c112200 con 0x7faa0c107240 2024-04-25T13:23:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa108f6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c112080 con 0x7faa0c19cf70 2024-04-25T13:23:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa108f6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c112380 con 0x7faa0c069350 2024-04-25T13:23:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa03fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 0x7faa0c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa03fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 0x7faa0c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54934/0 (socket says 172.21.15.72:54934) 2024-04-25T13:23:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa03fff640 1 -- 172.21.15.72:0/2507675777 learned_addr learned my addr 172.21.15.72:0/2507675777 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.405+0000 7faa0a59c640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa0c069350 0x7faa0c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa0ad9d640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c19cf70 0x7faa0c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa03fff640 1 -- 172.21.15.72:0/2507675777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa0c069350 msgr2=0x7faa0c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa03fff640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa0c069350 0x7faa0c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa03fff640 1 -- 172.21.15.72:0/2507675777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c19cf70 msgr2=0x7faa0c19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa03fff640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c19cf70 0x7faa0c19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa03fff640 1 -- 172.21.15.72:0/2507675777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c19da60 con 0x7faa0c107240 2024-04-25T13:23:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa03fff640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 0x7faa0c198c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400d770 tx=0x7fa9f400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa037fe640 1 -- 172.21.15.72:0/2507675777 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4017070 con 0x7faa0c107240 2024-04-25T13:23:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa037fe640 1 -- 172.21.15.72:0/2507675777 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f4010460 con 0x7faa0c107240 2024-04-25T13:23:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa037fe640 1 -- 172.21.15.72:0/2507675777 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f400f620 con 0x7faa0c107240 2024-04-25T13:23:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c19dd50 con 0x7faa0c107240 2024-04-25T13:23:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.406+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c1a54f0 con 0x7faa0c107240 2024-04-25T13:23:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.407+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c068d70 con 0x7faa0c107240 2024-04-25T13:23:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.408+0000 7faa037fe640 1 -- 172.21.15.72:0/2507675777 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9f401a070 con 0x7faa0c107240 2024-04-25T13:23:44.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.409+0000 7faa037fe640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa9f80a3da0 0x7fa9f80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:44.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.409+0000 7faa0a59c640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa9f80a3da0 0x7fa9f80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:44.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.409+0000 7faa037fe640 1 -- 172.21.15.72:0/2507675777 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fa9f40d0210 con 0x7faa0c107240 2024-04-25T13:23:44.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.409+0000 7faa0a59c640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa9f80a3da0 0x7fa9f80a6260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009990 tx=0x7fa9fc009110 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:44.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.412+0000 7faa037fe640 1 -- 172.21.15.72:0/2507675777 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9f409c830 con 0x7faa0c107240 2024-04-25T13:23:44.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.605+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faa0c069d00 con 0x7faa0c107240 2024-04-25T13:23:44.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.606+0000 7faa037fe640 1 -- 172.21.15.72:0/2507675777 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) ==== 81+0+944 (secure 0 0 0) 0x7fa9f409c130 con 0x7faa0c107240 2024-04-25T13:23:44.607 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/3348457169 2024-04-26T13:13:03.464543+0000 2024-04-25T13:23:44.607 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/4004940606 2024-04-26T13:13:03.464543+0000 2024-04-25T13:23:44.607 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3501290813 2024-04-26T13:13:03.464543+0000 2024-04-25T13:23:44.607 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/3348457169 2024-04-26T13:13:03.464543+0000 2024-04-25T13:23:44.607 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1633858068 2024-04-26T13:13:03.464543+0000 2024-04-25T13:23:44.607 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/2774697366 2024-04-26T13:04:07.045745+0000 2024-04-25T13:23:44.607 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/2774697366 2024-04-26T13:04:07.045745+0000 2024-04-25T13:23:44.607 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1368623402 2024-04-26T13:04:07.045745+0000 2024-04-25T13:23:44.608 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/338238237 2024-04-26T13:04:07.045745+0000 2024-04-25T13:23:44.608 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2198731116 2024-04-26T13:03:39.743288+0000 2024-04-25T13:23:44.608 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/830672933 2024-04-26T13:03:39.743288+0000 2024-04-25T13:23:44.608 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/2706752188 2024-04-26T13:03:39.743288+0000 2024-04-25T13:23:44.608 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1451045393 2024-04-26T13:03:39.743288+0000 2024-04-25T13:23:44.608 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1385198849 2024-04-26T13:04:07.045745+0000 2024-04-25T13:23:44.608 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2743588764 2024-04-26T13:13:03.464543+0000 2024-04-25T13:23:44.608 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/2706752188 2024-04-26T13:03:39.743288+0000 2024-04-25T13:23:44.609 INFO:teuthology.orchestra.run.smithi072.stderr:listed 16 entries 2024-04-25T13:23:44.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.607+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa9f80a3da0 msgr2=0x7fa9f80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.607+0000 7faa108f6640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa9f80a3da0 0x7fa9f80a6260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009990 tx=0x7fa9fc009110 comp rx=0 tx=0).stop 2024-04-25T13:23:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.607+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 msgr2=0x7faa0c198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.607+0000 7faa108f6640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 0x7faa0c198c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400d770 tx=0x7fa9f400db70 comp rx=0 tx=0).stop 2024-04-25T13:23:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.607+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 shutdown_connections 2024-04-25T13:23:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.608+0000 7faa108f6640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fa9f80a3da0 0x7fa9f80a6260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.608+0000 7faa108f6640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c19cf70 0x7faa0c19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.608+0000 7faa108f6640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa0c107240 0x7faa0c198c40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.608+0000 7faa108f6640 1 --2- 172.21.15.72:0/2507675777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa0c069350 0x7faa0c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.608+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 >> 172.21.15.72:0/2507675777 conn(0x7faa0c077cb0 msgr2=0x7faa0c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.608+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 shutdown_connections 2024-04-25T13:23:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:44.608+0000 7faa108f6640 1 -- 172.21.15.72:0/2507675777 wait complete. 2024-04-25T13:23:44.618 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:44 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/390139870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:44.618 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:44 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1789744936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:23:44.618 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:44 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1355433431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:23:44.619 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T13:23:44.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:23:44.620 DEBUG:teuthology.orchestra.run.smithi072:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T13:23:44.650 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:23:44.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:23:44.651 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:23:44.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:23:44.719 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-04-25T13:23:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:44 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/390139870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:44.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:44 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1789744936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:23:44.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:44 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1355433431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:23:44.774 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:23:44.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:23:44.775 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:23:44.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:44 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/390139870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:44.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:44 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1789744936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:23:44.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:44 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1355433431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:23:44.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:23:44.839 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2024-04-25T13:23:44.938 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T13:23:44.938 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-25T13:23:44.938 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:23:44.954 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=430 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/a3c9f6e2dbbf94440b8ef0ccc53eea91361907ae99e00ecbfe343b2eab074f4e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/a3c9f6e2dbbf94440b8ef0ccc53eea91361907ae99e00ecbfe343b2eab074f4e/diff,workdir=/var/lib/containers/storage/overlay/a3c9f6e2dbbf94440b8ef0ccc53eea91361907ae99e00ecbfe343b2eab074f4e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e695a7293d141d50c85d919d967b158e152e01705079fd1bb395c30fef200270/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/e695a7293d141d50c85d919d967b158e152e01705079fd1bb395c30fef200270/diff,workdir=/var/lib/containers/storage/overlay/e695a7293d141d50c85d919d967b158e152e01705079fd1bb395c30fef200270/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/74a25765e511dc64fdc89ae274347f104ba1e6e3c40fecd3f4c455af7cd68a54/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/74a25765e511dc64fdc89ae274347f104ba1e6e3c40fecd3f4c455af7cd68a54/diff,workdir=/var/lib/containers/storage/overlay/74a25765e511dc64fdc89ae274347f104ba1e6e3c40fecd3f4c455af7cd68a54/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/db05516ee2c56a0898e3af545df2f22b0f72bd49c2cfca0063dbc02cae8737ca/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/db05516ee2c56a0898e3af545df2f22b0f72bd49c2cfca0063dbc02cae8737ca/diff,workdir=/var/lib/containers/storage/overlay/db05516ee2c56a0898e3af545df2f22b0f72bd49c2cfca0063dbc02cae8737ca/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e4806ae283754df375cd8e8f702250eab8b8ce2761c44fe71442d90d8bc40579/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/e4806ae283754df375cd8e8f702250eab8b8ce2761c44fe71442d90d8bc40579/diff,workdir=/var/lib/containers/storage/overlay/e4806ae283754df375cd8e8f702250eab8b8ce2761c44fe71442d90d8bc40579/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/406a88e0836a82cf97188579c3e76c63f50cb41ed0dbe1d99c09ad2a04f4e07c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/406a88e0836a82cf97188579c3e76c63f50cb41ed0dbe1d99c09ad2a04f4e07c/diff,workdir=/var/lib/containers/storage/overlay/406a88e0836a82cf97188579c3e76c63f50cb41ed0dbe1d99c09ad2a04f4e07c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b237e79f5317373eee73ecaf41f6324e27d1e58afdc73fd65664469be1a65aa9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/b237e79f5317373eee73ecaf41f6324e27d1e58afdc73fd65664469be1a65aa9/diff,workdir=/var/lib/containers/storage/overlay/b237e79f5317373eee73ecaf41f6324e27d1e58afdc73fd65664469be1a65aa9/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/db53ce2d3e77863fab6f11486fa491b75c365a2eb0a81260fbc4cc11b32a297a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/db53ce2d3e77863fab6f11486fa491b75c365a2eb0a81260fbc4cc11b32a297a/diff,workdir=/var/lib/containers/storage/overlay/db53ce2d3e77863fab6f11486fa491b75c365a2eb0a81260fbc4cc11b32a297a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1c6d3d984c1e14c1ee8ee7e7ce91ff8896bec72f3b6f61c1135b90e5561af5e3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/1c6d3d984c1e14c1ee8ee7e7ce91ff8896bec72f3b6f61c1135b90e5561af5e3/diff,workdir=/var/lib/containers/storage/overlay/1c6d3d984c1e14c1ee8ee7e7ce91ff8896bec72f3b6f61c1135b90e5561af5e3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1b93b6c3a61f8d182a45c49dbcbb910067de784590246f51649fdb28f3a730cb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PM2XZZBJIENB4E7SDN2DK25IXX:/var/lib/containers/storage/overlay/l/UFK26RJIW4NERHYTLMFR2S2KML,upperdir=/var/lib/containers/storage/overlay/1b93b6c3a61f8d182a45c49dbcbb910067de784590246f51649fdb28f3a730cb/diff,workdir=/var/lib/containers/storage/overlay/1b93b6c3a61f8d182a45c49dbcbb910067de784590246f51649fdb28f3a730cb/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-25T13:23:44.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:23:44.954 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-04-25T13:23:45.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:23:45.010 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2024-04-25T13:23:45.075 INFO:teuthology.orchestra.run.smithi072.stderr:Cannot find device "ceph-brx" 2024-04-25T13:23:45.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:23:45.077 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T13:23:45.089 INFO:tasks.cephadm:Teardown begin 2024-04-25T13:23:45.089 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:23:45.089 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:45.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.215+0000 7f3600f5d640 1 -- 172.21.15.72:0/3988817828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1010a0 msgr2=0x7f35fc101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.215+0000 7f3600f5d640 1 --2- 172.21.15.72:0/3988817828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1010a0 0x7f35fc101480 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009900 tx=0x7f35f402a410 comp rx=0 tx=0).stop 2024-04-25T13:23:45.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.215+0000 7f3600f5d640 1 -- 172.21.15.72:0/3988817828 shutdown_connections 2024-04-25T13:23:45.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.215+0000 7f3600f5d640 1 --2- 172.21.15.72:0/3988817828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35fc106170 0x7f35fc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.215+0000 7f3600f5d640 1 --2- 172.21.15.72:0/3988817828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35fc101a50 0x7f35fc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.215+0000 7f3600f5d640 1 --2- 172.21.15.72:0/3988817828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1010a0 0x7f35fc101480 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.215+0000 7f3600f5d640 1 -- 172.21.15.72:0/3988817828 >> 172.21.15.72:0/3988817828 conn(0x7f35fc077f50 msgr2=0x7f35fc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:45.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.215+0000 7f3600f5d640 1 -- 172.21.15.72:0/3988817828 shutdown_connections 2024-04-25T13:23:45.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.215+0000 7f3600f5d640 1 -- 172.21.15.72:0/3988817828 wait complete. 2024-04-25T13:23:45.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f3600f5d640 1 Processor -- start 2024-04-25T13:23:45.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f3600f5d640 1 -- start start 2024-04-25T13:23:45.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f3600f5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35fc1010a0 0x7f35fc1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f3600f5d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc106170 0x7f35fc197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f3600f5d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35fc197e40 0x7f35fc1a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f3600f5d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1110b0 con 0x7f35fc106170 2024-04-25T13:23:45.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f3600f5d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc110f30 con 0x7f35fc197e40 2024-04-25T13:23:45.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f3600f5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc111230 con 0x7f35fc1010a0 2024-04-25T13:23:45.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35f9d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc106170 0x7f35fc197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35f9d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc106170 0x7f35fc197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54950/0 (socket says 172.21.15.72:54950) 2024-04-25T13:23:45.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35f9d9b640 1 -- 172.21.15.72:0/77242333 learned_addr learned my addr 172.21.15.72:0/77242333 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:45.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35fa59c640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35fc1010a0 0x7f35fc1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35fad9d640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35fc197e40 0x7f35fc1a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35f9d9b640 1 -- 172.21.15.72:0/77242333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35fc1010a0 msgr2=0x7f35fc1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35f9d9b640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35fc1010a0 0x7f35fc1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35f9d9b640 1 -- 172.21.15.72:0/77242333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35fc197e40 msgr2=0x7f35fc1a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35f9d9b640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35fc197e40 0x7f35fc1a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35f9d9b640 1 -- 172.21.15.72:0/77242333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc1a7000 con 0x7f35fc106170 2024-04-25T13:23:45.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35f9d9b640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc106170 0x7f35fc197900 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f35f000eab0 tx=0x7f35f000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:45.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.216+0000 7f35eb7fe640 1 -- 172.21.15.72:0/77242333 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f000cde0 con 0x7f35fc106170 2024-04-25T13:23:45.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.217+0000 7f35eb7fe640 1 -- 172.21.15.72:0/77242333 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f0011920 con 0x7f35fc106170 2024-04-25T13:23:45.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.217+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35fc1a72f0 con 0x7f35fc106170 2024-04-25T13:23:45.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.217+0000 7f35eb7fe640 1 -- 172.21.15.72:0/77242333 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f001a9d0 con 0x7f35fc106170 2024-04-25T13:23:45.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.217+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35fc1a7840 con 0x7f35fc106170 2024-04-25T13:23:45.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.217+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35fc068d70 con 0x7f35fc106170 2024-04-25T13:23:45.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.219+0000 7f35eb7fe640 1 -- 172.21.15.72:0/77242333 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f35f001ab70 con 0x7f35fc106170 2024-04-25T13:23:45.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.219+0000 7f35eb7fe640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f35e40a3d20 0x7f35e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.219+0000 7f35fa59c640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f35e40a3d20 0x7f35e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.219+0000 7f35eb7fe640 1 -- 172.21.15.72:0/77242333 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f35f00d1040 con 0x7f35fc106170 2024-04-25T13:23:45.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.220+0000 7f35fa59c640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f35e40a3d20 0x7f35e40a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f35f4031440 tx=0x7f35f40095f0 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:45.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.223+0000 7f35eb7fe640 1 -- 172.21.15.72:0/77242333 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35f0016030 con 0x7f35fc106170 2024-04-25T13:23:45.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.443+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35fc0696d0 con 0x7f35fc106170 2024-04-25T13:23:45.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.444+0000 7f35eb7fe640 1 -- 172.21.15.72:0/77242333 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7f35fc0696d0 con 0x7f35fc106170 2024-04-25T13:23:45.445 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:23:45.445 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:11:04.601639+0000","modified":"2024-04-25T13:13:02.144260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24493,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6839/1444196308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1444196308},{"type":"v1","addr":"172.21.15.164:6839","nonce":1444196308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24478,24493]},"id":1}]} 2024-04-25T13:23:45.445 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 21 2024-04-25T13:23:45.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f35e40a3d20 msgr2=0x7f35e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f35e40a3d20 0x7f35e40a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f35f4031440 tx=0x7f35f40095f0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc106170 msgr2=0x7f35fc197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc106170 0x7f35fc197900 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f35f000eab0 tx=0x7f35f000ef70 comp rx=0 tx=0).stop 2024-04-25T13:23:45.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 shutdown_connections 2024-04-25T13:23:45.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f35e40a3d20 0x7f35e40a61e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35fc197e40 0x7f35fc1a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc106170 0x7f35fc197900 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 --2- 172.21.15.72:0/77242333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35fc1010a0 0x7f35fc1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 >> 172.21.15.72:0/77242333 conn(0x7f35fc077f50 msgr2=0x7f35fc0ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:45.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 shutdown_connections 2024-04-25T13:23:45.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.446+0000 7f3600f5d640 1 -- 172.21.15.72:0/77242333 wait complete. 2024-04-25T13:23:45.458 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:45.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.555+0000 7fe962fa1640 1 -- 172.21.15.72:0/1997086476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c101a50 msgr2=0x7fe95c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.555+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1997086476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c101a50 0x7fe95c105a90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009900 tx=0x7fe94c02a410 comp rx=0 tx=0).stop 2024-04-25T13:23:45.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.555+0000 7fe962fa1640 1 -- 172.21.15.72:0/1997086476 shutdown_connections 2024-04-25T13:23:45.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.555+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1997086476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c106170 0x7fe95c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.555+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1997086476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c101a50 0x7fe95c105a90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.555+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1997086476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c1010a0 0x7fe95c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 -- 172.21.15.72:0/1997086476 >> 172.21.15.72:0/1997086476 conn(0x7fe95c077f50 msgr2=0x7fe95c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:45.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 -- 172.21.15.72:0/1997086476 shutdown_connections 2024-04-25T13:23:45.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 -- 172.21.15.72:0/1997086476 wait complete. 2024-04-25T13:23:45.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 Processor -- start 2024-04-25T13:23:45.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 -- start start 2024-04-25T13:23:45.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c1010a0 0x7fe95c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c106170 0x7fe95c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c19a080 0x7fe95c1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c111110 con 0x7fe95c106170 2024-04-25T13:23:45.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c110f90 con 0x7fe95c19a080 2024-04-25T13:23:45.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe962fa1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c111290 con 0x7fe95c1010a0 2024-04-25T13:23:45.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c106170 0x7fe95c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c106170 0x7fe95c199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43516/0 (socket says 172.21.15.72:43516) 2024-04-25T13:23:45.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.556+0000 7fe95bfff640 1 -- 172.21.15.72:0/1847021135 learned_addr learned my addr 172.21.15.72:0/1847021135 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:45.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe96153e640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c19a080 0x7fe95c1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe960d3d640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c1010a0 0x7fe95c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe95bfff640 1 -- 172.21.15.72:0/1847021135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c1010a0 msgr2=0x7fe95c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe95bfff640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c1010a0 0x7fe95c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe95bfff640 1 -- 172.21.15.72:0/1847021135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c19a080 msgr2=0x7fe95c1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe95bfff640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c19a080 0x7fe95c1a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe95bfff640 1 -- 172.21.15.72:0/1847021135 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95c1a4e30 con 0x7fe95c106170 2024-04-25T13:23:45.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe95bfff640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c106170 0x7fe95c199af0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe94c031a90 tx=0x7fe94c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:45.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe959ffb640 1 -- 172.21.15.72:0/1847021135 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c03f030 con 0x7fe95c106170 2024-04-25T13:23:45.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95c1a50c0 con 0x7fe95c106170 2024-04-25T13:23:45.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe959ffb640 1 -- 172.21.15.72:0/1847021135 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94c003cb0 con 0x7fe95c106170 2024-04-25T13:23:45.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe959ffb640 1 -- 172.21.15.72:0/1847021135 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c035070 con 0x7fe95c106170 2024-04-25T13:23:45.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.557+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95c1a5600 con 0x7fe95c106170 2024-04-25T13:23:45.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.558+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95c068d70 con 0x7fe95c106170 2024-04-25T13:23:45.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.559+0000 7fe959ffb640 1 -- 172.21.15.72:0/1847021135 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe94c042030 con 0x7fe95c106170 2024-04-25T13:23:45.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.560+0000 7fe959ffb640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe9480a3d20 0x7fe9480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.560+0000 7fe960d3d640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe9480a3d20 0x7fe9480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.560+0000 7fe959ffb640 1 -- 172.21.15.72:0/1847021135 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7fe94c0f0e80 con 0x7fe95c106170 2024-04-25T13:23:45.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.560+0000 7fe960d3d640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe9480a3d20 0x7fe9480a61e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe954009f20 tx=0x7fe9540094f0 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:45.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.564+0000 7fe959ffb640 1 -- 172.21.15.72:0/1847021135 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe94c0bd500 con 0x7fe95c106170 2024-04-25T13:23:45.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:45 smithi193 ceph-mon[32743]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:45.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:45 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2507675777' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:23:45.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:45 smithi072 ceph-mon[29199]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:45.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:45 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2507675777' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:23:45.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.782+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe95c1093a0 con 0x7fe95c106170 2024-04-25T13:23:45.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.783+0000 7fe959ffb640 1 -- 172.21.15.72:0/1847021135 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7fe94c0bce00 con 0x7fe95c106170 2024-04-25T13:23:45.783 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:23:45.784 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3145624420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3145624420},{"type":"v1","addr":"172.21.15.72:6835","nonce":3145624420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1560520314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1560520314},{"type":"v1","addr":"172.21.15.72:6837","nonce":1560520314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3204730698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3204730698},{"type":"v1","addr":"172.21.15.72:6839","nonce":3204730698}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/882585461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":882585461},{"type":"v1","addr":"172.21.15.72:6841","nonce":882585461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1516437385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1516437385},{"type":"v1","addr":"172.21.15.164:6833","nonce":1516437385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3064383723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3064383723},{"type":"v1","addr":"172.21.15.193:6833","nonce":3064383723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:11:04.601639+0000","modified":"2024-04-25T13:13:02.144260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24493,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3356589609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3356589609},{"type":"v1","addr":"172.21.15.164:6835","nonce":3356589609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/1113925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1113925217},{"type":"v1","addr":"172.21.15.193:6835","nonce":1113925217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/1335235896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1335235896},{"type":"v1","addr":"172.21.15.164:6837","nonce":1335235896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.193:6837/1407296685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1407296685},{"type":"v1","addr":"172.21.15.193:6837","nonce":1407296685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/2585421276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":2585421276},{"type":"v1","addr":"172.21.15.193:6839","nonce":2585421276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6839/1444196308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1444196308},{"type":"v1","addr":"172.21.15.164:6839","nonce":1444196308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24478,24493]},"id":1}]} 2024-04-25T13:23:45.784 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 21 2024-04-25T13:23:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe9480a3d20 msgr2=0x7fe9480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe9480a3d20 0x7fe9480a61e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe954009f20 tx=0x7fe9540094f0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c106170 msgr2=0x7fe95c199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c106170 0x7fe95c199af0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe94c031a90 tx=0x7fe94c004270 comp rx=0 tx=0).stop 2024-04-25T13:23:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 shutdown_connections 2024-04-25T13:23:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7fe9480a3d20 0x7fe9480a61e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c19a080 0x7fe95c1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe95c106170 0x7fe95c199af0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 --2- 172.21.15.72:0/1847021135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c1010a0 0x7fe95c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 >> 172.21.15.72:0/1847021135 conn(0x7fe95c077f50 msgr2=0x7fe95c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:45.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 shutdown_connections 2024-04-25T13:23:45.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.785+0000 7fe962fa1640 1 -- 172.21.15.72:0/1847021135 wait complete. 2024-04-25T13:23:45.796 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:23:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:45 smithi164 ceph-mon[32418]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:23:45.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:45 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2507675777' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:23:45.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.893+0000 7f124acea640 1 -- 172.21.15.72:0/857762018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1244106190 msgr2=0x7f124410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.893+0000 7f124acea640 1 --2- 172.21.15.72:0/857762018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1244106190 0x7f124410dd60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f123c009ca0 tx=0x7f123c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.893+0000 7f124acea640 1 -- 172.21.15.72:0/857762018 shutdown_connections 2024-04-25T13:23:45.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.893+0000 7f124acea640 1 --2- 172.21.15.72:0/857762018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1244106190 0x7f124410dd60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.893+0000 7f124acea640 1 --2- 172.21.15.72:0/857762018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1244101a70 0x7f1244105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.893+0000 7f124acea640 1 --2- 172.21.15.72:0/857762018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12441010c0 0x7f12441014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.893+0000 7f124acea640 1 -- 172.21.15.72:0/857762018 >> 172.21.15.72:0/857762018 conn(0x7f12440771d0 msgr2=0x7f12440775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:45.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.893+0000 7f124acea640 1 -- 172.21.15.72:0/857762018 shutdown_connections 2024-04-25T13:23:45.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.893+0000 7f124acea640 1 -- 172.21.15.72:0/857762018 wait complete. 2024-04-25T13:23:45.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f124acea640 1 Processor -- start 2024-04-25T13:23:45.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f124acea640 1 -- start start 2024-04-25T13:23:45.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f124acea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12441010c0 0x7f12441986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f124acea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1244106190 0x7f1244198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f124acea640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f124419cf80 0x7f124419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f124acea640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1244111100 con 0x7f124419cf80 2024-04-25T13:23:45.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f124acea640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1244110f80 con 0x7f1244106190 2024-04-25T13:23:45.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f124acea640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1244111280 con 0x7f12441010c0 2024-04-25T13:23:45.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f1249287640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f124419cf80 0x7f124419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f1249287640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f124419cf80 0x7f124419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43530/0 (socket says 172.21.15.72:43530) 2024-04-25T13:23:45.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f1249287640 1 -- 172.21.15.72:0/2337848750 learned_addr learned my addr 172.21.15.72:0/2337848750 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:45.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f1249287640 1 -- 172.21.15.72:0/2337848750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12441010c0 msgr2=0x7f12441986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f1248a86640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12441010c0 0x7f12441986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f1249287640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12441010c0 0x7f12441986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f1249287640 1 -- 172.21.15.72:0/2337848750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1244106190 msgr2=0x7f1244198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f1249287640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1244106190 0x7f1244198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.894+0000 7f1249287640 1 -- 172.21.15.72:0/2337848750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124419da70 con 0x7f124419cf80 2024-04-25T13:23:45.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.895+0000 7f1249287640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f124419cf80 0x7f124419d430 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f123c005cc0 tx=0x7f123c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:45.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.895+0000 7f1239ffb640 1 -- 172.21.15.72:0/2337848750 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c003bf0 con 0x7f124419cf80 2024-04-25T13:23:45.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.895+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124419dd00 con 0x7f124419cf80 2024-04-25T13:23:45.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.895+0000 7f1239ffb640 1 -- 172.21.15.72:0/2337848750 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123c003d90 con 0x7f124419cf80 2024-04-25T13:23:45.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.895+0000 7f1239ffb640 1 -- 172.21.15.72:0/2337848750 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c02fbc0 con 0x7f124419cf80 2024-04-25T13:23:45.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.895+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12441a5520 con 0x7f124419cf80 2024-04-25T13:23:45.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.896+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12440691c0 con 0x7f124419cf80 2024-04-25T13:23:45.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.897+0000 7f1239ffb640 1 -- 172.21.15.72:0/2337848750 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f123c02fd60 con 0x7f124419cf80 2024-04-25T13:23:45.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.898+0000 7f1239ffb640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f122c0a3d20 0x7f122c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.898+0000 7f1248a86640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f122c0a3d20 0x7f122c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.898+0000 7f1239ffb640 1 -- 172.21.15.72:0/2337848750 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7f123c0f1fe0 con 0x7f124419cf80 2024-04-25T13:23:45.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.898+0000 7f1248a86640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f122c0a3d20 0x7f122c0a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1240005fd0 tx=0x7f1240005850 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:45.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:45.902+0000 7f1239ffb640 1 -- 172.21.15.72:0/2337848750 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f123c0be890 con 0x7f124419cf80 2024-04-25T13:23:46.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.080+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1244101a70 con 0x7f124419cf80 2024-04-25T13:23:46.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.081+0000 7f1239ffb640 1 -- 172.21.15.72:0/2337848750 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23208 (secure 0 0 0) 0x7f123c0be190 con 0x7f124419cf80 2024-04-25T13:23:46.082 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-25T13:23:46.082 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":99,"fsid":"d8ca43ea-0303-11ef-bc93-c7b262605968","created":"2024-04-25T13:02:57.590034+0000","modified":"2024-04-25T13:23:17.416503+0000","last_up_change":"2024-04-25T13:10:14.482039+0000","last_in_change":"2024-04-25T13:09:54.858041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:06:33.150709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:11:02.352089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:11:03.565033+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86128e0d-c8b1-4771-a222-d42316fe3351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6803","nonce":2197040660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6805","nonce":2197040660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6809","nonce":2197040660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":2197040660},{"type":"v1","addr":"172.21.15.72:6807","nonce":2197040660}]},"public_addr":"172.21.15.72:6803/2197040660","cluster_addr":"172.21.15.72:6805/2197040660","heartbeat_back_addr":"172.21.15.72:6809/2197040660","heartbeat_front_addr":"172.21.15.72:6807/2197040660","state":["exists","up"]},{"osd":1,"uuid":"a2389db1-f136-4e7a-a5c9-bc63d97b938c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6801","nonce":4084359931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6803","nonce":4084359931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6807","nonce":4084359931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4084359931},{"type":"v1","addr":"172.21.15.164:6805","nonce":4084359931}]},"public_addr":"172.21.15.164:6801/4084359931","cluster_addr":"172.21.15.164:6803/4084359931","heartbeat_back_addr":"172.21.15.164:6807/4084359931","heartbeat_front_addr":"172.21.15.164:6805/4084359931","state":["exists","up"]},{"osd":2,"uuid":"778a3d43-534a-4aca-b391-07b8b9e045a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6801","nonce":1367843850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6803","nonce":1367843850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6807","nonce":1367843850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1367843850},{"type":"v1","addr":"172.21.15.193:6805","nonce":1367843850}]},"public_addr":"172.21.15.193:6801/1367843850","cluster_addr":"172.21.15.193:6803/1367843850","heartbeat_back_addr":"172.21.15.193:6807/1367843850","heartbeat_front_addr":"172.21.15.193:6805/1367843850","state":["exists","up"]},{"osd":3,"uuid":"2134ce99-2a05-49fd-8653-81180a5c199a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6811","nonce":840491303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6813","nonce":840491303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6817","nonce":840491303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":840491303},{"type":"v1","addr":"172.21.15.72:6815","nonce":840491303}]},"public_addr":"172.21.15.72:6811/840491303","cluster_addr":"172.21.15.72:6813/840491303","heartbeat_back_addr":"172.21.15.72:6817/840491303","heartbeat_front_addr":"172.21.15.72:6815/840491303","state":["exists","up"]},{"osd":4,"uuid":"79d3bd39-e0cb-403c-8911-1a633198aa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6809","nonce":1819655779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6811","nonce":1819655779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6815","nonce":1819655779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1819655779},{"type":"v1","addr":"172.21.15.164:6813","nonce":1819655779}]},"public_addr":"172.21.15.164:6809/1819655779","cluster_addr":"172.21.15.164:6811/1819655779","heartbeat_back_addr":"172.21.15.164:6815/1819655779","heartbeat_front_addr":"172.21.15.164:6813/1819655779","state":["exists","up"]},{"osd":5,"uuid":"c64e72ba-dfe6-43fe-ba1c-4819ac7bc211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6809","nonce":2502247683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6811","nonce":2502247683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6815","nonce":2502247683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2502247683},{"type":"v1","addr":"172.21.15.193:6813","nonce":2502247683}]},"public_addr":"172.21.15.193:6809/2502247683","cluster_addr":"172.21.15.193:6811/2502247683","heartbeat_back_addr":"172.21.15.193:6815/2502247683","heartbeat_front_addr":"172.21.15.193:6813/2502247683","state":["exists","up"]},{"osd":6,"uuid":"e4aa6d06-f6dd-4e6e-a6cf-58bb6cff57dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6819","nonce":3792057765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6821","nonce":3792057765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6825","nonce":3792057765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3792057765},{"type":"v1","addr":"172.21.15.72:6823","nonce":3792057765}]},"public_addr":"172.21.15.72:6819/3792057765","cluster_addr":"172.21.15.72:6821/3792057765","heartbeat_back_addr":"172.21.15.72:6825/3792057765","heartbeat_front_addr":"172.21.15.72:6823/3792057765","state":["exists","up"]},{"osd":7,"uuid":"7ca180ab-29ed-4109-bb1d-f7d19cb086b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6817","nonce":3610097866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6819","nonce":3610097866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6823","nonce":3610097866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3610097866},{"type":"v1","addr":"172.21.15.164:6821","nonce":3610097866}]},"public_addr":"172.21.15.164:6817/3610097866","cluster_addr":"172.21.15.164:6819/3610097866","heartbeat_back_addr":"172.21.15.164:6823/3610097866","heartbeat_front_addr":"172.21.15.164:6821/3610097866","state":["exists","up"]},{"osd":8,"uuid":"dd29481d-1fbb-4b9b-b066-f85db8310162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6817","nonce":2354460844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6819","nonce":2354460844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6823","nonce":2354460844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2354460844},{"type":"v1","addr":"172.21.15.193:6821","nonce":2354460844}]},"public_addr":"172.21.15.193:6817/2354460844","cluster_addr":"172.21.15.193:6819/2354460844","heartbeat_back_addr":"172.21.15.193:6823/2354460844","heartbeat_front_addr":"172.21.15.193:6821/2354460844","state":["exists","up"]},{"osd":9,"uuid":"10681718-08e5-4452-845f-75a2299d281f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6827","nonce":4174455071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6829","nonce":4174455071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6833","nonce":4174455071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":4174455071},{"type":"v1","addr":"172.21.15.72:6831","nonce":4174455071}]},"public_addr":"172.21.15.72:6827/4174455071","cluster_addr":"172.21.15.72:6829/4174455071","heartbeat_back_addr":"172.21.15.72:6833/4174455071","heartbeat_front_addr":"172.21.15.72:6831/4174455071","state":["exists","up"]},{"osd":10,"uuid":"fcbcf574-0a98-411a-ac22-fe6520a5c1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6825","nonce":1853012500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6827","nonce":1853012500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6831","nonce":1853012500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1853012500},{"type":"v1","addr":"172.21.15.164:6829","nonce":1853012500}]},"public_addr":"172.21.15.164:6825/1853012500","cluster_addr":"172.21.15.164:6827/1853012500","heartbeat_back_addr":"172.21.15.164:6831/1853012500","heartbeat_front_addr":"172.21.15.164:6829/1853012500","state":["exists","up"]},{"osd":11,"uuid":"36fabfaa-4071-45f7-8d95-1a989dcba56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6825","nonce":2763093490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6827","nonce":2763093490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6831","nonce":2763093490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2763093490},{"type":"v1","addr":"172.21.15.193:6829","nonce":2763093490}]},"public_addr":"172.21.15.193:6825/2763093490","cluster_addr":"172.21.15.193:6827/2763093490","heartbeat_back_addr":"172.21.15.193:6831/2763093490","heartbeat_front_addr":"172.21.15.193:6829/2763093490","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:05:41.754768+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:06.320096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:29.854931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:06:54.266727+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:18.731612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:07:42.668369+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:08.253924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:32.715883+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:08:57.456134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:23.017200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:09:47.534496+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:10:12.966670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/3348457169":"2024-04-26T13:13:03.464543+0000","172.21.15.72:0/4004940606":"2024-04-26T13:13:03.464543+0000","172.21.15.72:0/3501290813":"2024-04-26T13:13:03.464543+0000","172.21.15.72:6800/3348457169":"2024-04-26T13:13:03.464543+0000","172.21.15.72:0/1633858068":"2024-04-26T13:13:03.464543+0000","172.21.15.72:6801/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:6800/2774697366":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/1368623402":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/338238237":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2198731116":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/830672933":"2024-04-26T13:03:39.743288+0000","172.21.15.72:6800/2706752188":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1451045393":"2024-04-26T13:03:39.743288+0000","172.21.15.72:0/1385198849":"2024-04-26T13:04:07.045745+0000","172.21.15.72:0/2743588764":"2024-04-26T13:13:03.464543+0000","172.21.15.72:6801/2706752188":"2024-04-26T13:03:39.743288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:23:46.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f122c0a3d20 msgr2=0x7f122c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f122c0a3d20 0x7f122c0a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1240005fd0 tx=0x7f1240005850 comp rx=0 tx=0).stop 2024-04-25T13:23:46.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f124419cf80 msgr2=0x7f124419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f124419cf80 0x7f124419d430 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f123c005cc0 tx=0x7f123c02f380 comp rx=0 tx=0).stop 2024-04-25T13:23:46.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 shutdown_connections 2024-04-25T13:23:46.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f122c0a3d20 0x7f122c0a61e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f124419cf80 0x7f124419d430 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1244106190 0x7f1244198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 --2- 172.21.15.72:0/2337848750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12441010c0 0x7f12441986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 >> 172.21.15.72:0/2337848750 conn(0x7f12440771d0 msgr2=0x7f124406a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:46.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 shutdown_connections 2024-04-25T13:23:46.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.083+0000 7f124acea640 1 -- 172.21.15.72:0/2337848750 wait complete. 2024-04-25T13:23:46.095 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:23:46.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 -- 172.21.15.72:0/2084010489 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c103180 msgr2=0x7feb0c103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 --2- 172.21.15.72:0/2084010489 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c103180 0x7feb0c103560 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feb00009900 tx=0x7feb0002a480 comp rx=0 tx=0).stop 2024-04-25T13:23:46.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 -- 172.21.15.72:0/2084010489 shutdown_connections 2024-04-25T13:23:46.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 --2- 172.21.15.72:0/2084010489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb0c10c550 0x7feb0c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 --2- 172.21.15.72:0/2084010489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb0c103aa0 0x7feb0c10bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 --2- 172.21.15.72:0/2084010489 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c103180 0x7feb0c103560 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 -- 172.21.15.72:0/2084010489 >> 172.21.15.72:0/2084010489 conn(0x7feb0c0771d0 msgr2=0x7feb0c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:46.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 -- 172.21.15.72:0/2084010489 shutdown_connections 2024-04-25T13:23:46.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 -- 172.21.15.72:0/2084010489 wait complete. 2024-04-25T13:23:46.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 Processor -- start 2024-04-25T13:23:46.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.194+0000 7feb12c74640 1 -- start start 2024-04-25T13:23:46.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb12c74640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb0c103180 0x7feb0c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:46.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb12c74640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb0c103aa0 0x7feb0c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:46.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb12c74640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c10c550 0x7feb0c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:46.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb12c74640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c112180 con 0x7feb0c10c550 2024-04-25T13:23:46.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb12c74640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c112000 con 0x7feb0c103180 2024-04-25T13:23:46.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb12c74640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c112300 con 0x7feb0c103aa0 2024-04-25T13:23:46.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb11211640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c10c550 0x7feb0c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:46.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb11211640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c10c550 0x7feb0c19d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43548/0 (socket says 172.21.15.72:43548) 2024-04-25T13:23:46.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb11211640 1 -- 172.21.15.72:0/3146153353 learned_addr learned my addr 172.21.15.72:0/3146153353 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:46.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb0bfff640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb0c103aa0 0x7feb0c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:46.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb10a10640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb0c103180 0x7feb0c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:46.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb11211640 1 -- 172.21.15.72:0/3146153353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb0c103aa0 msgr2=0x7feb0c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb11211640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb0c103aa0 0x7feb0c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb11211640 1 -- 172.21.15.72:0/3146153353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb0c103180 msgr2=0x7feb0c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb11211640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb0c103180 0x7feb0c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb11211640 1 -- 172.21.15.72:0/3146153353 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c19d740 con 0x7feb0c10c550 2024-04-25T13:23:46.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb0bfff640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb0c103aa0 0x7feb0c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:23:46.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb11211640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c10c550 0x7feb0c19d010 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feb0400cad0 tx=0x7feb0400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:46.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb09ffb640 1 -- 172.21.15.72:0/3146153353 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0400ae20 con 0x7feb0c10c550 2024-04-25T13:23:46.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb09ffb640 1 -- 172.21.15.72:0/3146153353 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb04005020 con 0x7feb0c10c550 2024-04-25T13:23:46.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c19da30 con 0x7feb0c10c550 2024-04-25T13:23:46.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.195+0000 7feb09ffb640 1 -- 172.21.15.72:0/3146153353 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0400f6b0 con 0x7feb0c10c550 2024-04-25T13:23:46.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.196+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c1a52c0 con 0x7feb0c10c550 2024-04-25T13:23:46.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.196+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0c06add0 con 0x7feb0c10c550 2024-04-25T13:23:46.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.197+0000 7feb09ffb640 1 -- 172.21.15.72:0/3146153353 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb040051d0 con 0x7feb0c10c550 2024-04-25T13:23:46.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.198+0000 7feb09ffb640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7feafc0a3da0 0x7feafc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:46.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.198+0000 7feb09ffb640 1 -- 172.21.15.72:0/3146153353 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8550+0+0 (secure 0 0 0) 0x7feb040d0870 con 0x7feb0c10c550 2024-04-25T13:23:46.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.198+0000 7feb10a10640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7feafc0a3da0 0x7feafc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:46.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.198+0000 7feb10a10640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7feafc0a3da0 0x7feafc0a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb00031440 tx=0x7feb000095f0 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:46.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.202+0000 7feb09ffb640 1 -- 172.21.15.72:0/3146153353 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb04014030 con 0x7feb0c10c550 2024-04-25T13:23:46.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.417+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7feb0c199aa0 con 0x7feb0c10c550 2024-04-25T13:23:46.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.419+0000 7feb09ffb640 1 -- 172.21.15.72:0/3146153353 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7feb0409cef0 con 0x7feb0c10c550 2024-04-25T13:23:46.420 INFO:teuthology.orchestra.run.smithi072.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:23:46.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7feafc0a3da0 msgr2=0x7feafc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7feafc0a3da0 0x7feafc0a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb00031440 tx=0x7feb000095f0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c10c550 msgr2=0x7feb0c19d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c10c550 0x7feb0c19d010 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feb0400cad0 tx=0x7feb0400cf90 comp rx=0 tx=0).stop 2024-04-25T13:23:46.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 shutdown_connections 2024-04-25T13:23:46.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7feafc0a3da0 0x7feafc0a6260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb0c10c550 0x7feb0c19d010 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb0c103aa0 0x7feb0c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 --2- 172.21.15.72:0/3146153353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb0c103180 0x7feb0c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 >> 172.21.15.72:0/3146153353 conn(0x7feb0c0771d0 msgr2=0x7feb0c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:46.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 shutdown_connections 2024-04-25T13:23:46.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.422+0000 7feb12c74640 1 -- 172.21.15.72:0/3146153353 wait complete. 2024-04-25T13:23:46.434 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:23:46.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.535+0000 7f69657ad640 1 -- 172.21.15.72:0/1442836730 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 msgr2=0x7f6960100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.535+0000 7f69657ad640 1 --2- 172.21.15.72:0/1442836730 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 0x7f6960100ac0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6950009900 tx=0x7f695002a410 comp rx=0 tx=0).stop 2024-04-25T13:23:46.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.535+0000 7f69657ad640 1 -- 172.21.15.72:0/1442836730 shutdown_connections 2024-04-25T13:23:46.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.535+0000 7f69657ad640 1 --2- 172.21.15.72:0/1442836730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69601072f0 0x7f696010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.535+0000 7f69657ad640 1 --2- 172.21.15.72:0/1442836730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6960101090 0x7f6960106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.535+0000 7f69657ad640 1 --2- 172.21.15.72:0/1442836730 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 0x7f6960100ac0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.535+0000 7f69657ad640 1 -- 172.21.15.72:0/1442836730 >> 172.21.15.72:0/1442836730 conn(0x7f69600773e0 msgr2=0x7f69600fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:46.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.535+0000 7f69657ad640 1 -- 172.21.15.72:0/1442836730 shutdown_connections 2024-04-25T13:23:46.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.535+0000 7f69657ad640 1 -- 172.21.15.72:0/1442836730 wait complete. 2024-04-25T13:23:46.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f69657ad640 1 Processor -- start 2024-04-25T13:23:46.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f69657ad640 1 -- start start 2024-04-25T13:23:46.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f69657ad640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 0x7f696019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:46.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f69657ad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69601072f0 0x7f696019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:46.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f69657ad640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6960198600 0x7f6960198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:46.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f69657ad640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6960112200 con 0x7f69601006e0 2024-04-25T13:23:46.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f69657ad640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6960112080 con 0x7f69601072f0 2024-04-25T13:23:46.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f69657ad640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6960112380 con 0x7f6960198600 2024-04-25T13:23:46.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f695effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 0x7f696019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:46.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f695effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 0x7f696019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43560/0 (socket says 172.21.15.72:43560) 2024-04-25T13:23:46.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f695effd640 1 -- 172.21.15.72:0/1035740988 learned_addr learned my addr 172.21.15.72:0/1035740988 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:46.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f695f7fe640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6960198600 0x7f6960198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:46.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.536+0000 7f695e7fc640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69601072f0 0x7f696019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:46.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f695effd640 1 -- 172.21.15.72:0/1035740988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6960198600 msgr2=0x7f6960198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f695effd640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6960198600 0x7f6960198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f695effd640 1 -- 172.21.15.72:0/1035740988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69601072f0 msgr2=0x7f696019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f695effd640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69601072f0 0x7f696019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f695effd640 1 -- 172.21.15.72:0/1035740988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6960199120 con 0x7f69601006e0 2024-04-25T13:23:46.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f695effd640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 0x7f696019e450 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6950005940 tx=0x7f6950004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:46.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f6947fff640 1 -- 172.21.15.72:0/1035740988 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695003f030 con 0x7f69601006e0 2024-04-25T13:23:46.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69601993b0 con 0x7f69601006e0 2024-04-25T13:23:46.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f6947fff640 1 -- 172.21.15.72:0/1035740988 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6950003aa0 con 0x7f69601006e0 2024-04-25T13:23:46.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f6947fff640 1 -- 172.21.15.72:0/1035740988 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950035070 con 0x7f69601006e0 2024-04-25T13:23:46.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.537+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69601a55a0 con 0x7f69601006e0 2024-04-25T13:23:46.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.538+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69600691c0 con 0x7f69601006e0 2024-04-25T13:23:46.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.539+0000 7f6947fff640 1 -- 172.21.15.72:0/1035740988 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6950042030 con 0x7f69601006e0 2024-04-25T13:23:46.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.540+0000 7f6947fff640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f69480a3d20 0x7f69480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:46.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.540+0000 7f695e7fc640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f69480a3d20 0x7f69480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:46.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.540+0000 7f6947fff640 1 -- 172.21.15.72:0/1035740988 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8764+0+0 (secure 0 0 0) 0x7f69500f0f30 con 0x7f69601006e0 2024-04-25T13:23:46.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.541+0000 7f695e7fc640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f69480a3d20 0x7f69480a61e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69600ffb50 tx=0x7f6954005f90 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:46.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.545+0000 7f6947fff640 1 -- 172.21.15.72:0/1035740988 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69500bce90 con 0x7f69601006e0 2024-04-25T13:23:46.732 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c[62817]: ignoring --setuser ceph since I am not root 2024-04-25T13:23:46.732 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c[62817]: ignoring --setgroup ceph since I am not root 2024-04-25T13:23:46.732 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mds[62821]: -- 172.21.15.193:0/602773745 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d55d37a40 con 0x560d55d0b000 2024-04-25T13:23:46.732 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c[62817]: starting mds.c at 2024-04-25T13:23:46.733 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f[62501]: ignoring --setuser ceph since I am not root 2024-04-25T13:23:46.733 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f[62501]: ignoring --setgroup ceph since I am not root 2024-04-25T13:23:46.733 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mds[62505]: -- 172.21.15.193:0/4168665153 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5584cd039a40 con 0x5584cd00c800 2024-04-25T13:23:46.733 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f[62501]: starting mds.f at 2024-04-25T13:23:46.734 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i[63140]: ignoring --setuser ceph since I am not root 2024-04-25T13:23:46.734 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i[63140]: ignoring --setgroup ceph since I am not root 2024-04-25T13:23:46.734 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mds[63144]: -- 172.21.15.193:0/2324190563 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a75d8fa40 con 0x563a75d62800 2024-04-25T13:23:46.734 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i[63140]: starting mds.i at 2024-04-25T13:23:46.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/77242333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:46.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1847021135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:46.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2337848750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3146153353' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:23:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mon[32743]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:23:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mon[32743]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:23:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3146153353' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:23:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mon[32743]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:23:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:46 smithi193 ceph-mon[32743]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T13:23:46.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:46.763+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6960101090 con 0x7f69601006e0 2024-04-25T13:23:46.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a[29195]: 2024-04-25T13:23:46.418+0000 7f7357b8e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:23:46.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/77242333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:46.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1847021135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:46.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2337848750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:46.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3146153353' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:23:46.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-mon[29199]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:23:46.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-mon[29199]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:23:46.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3146153353' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:23:46.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-mon[29199]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:23:46.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:46 smithi072 ceph-mon[29199]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T13:23:46.807 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b[62256]: ignoring --setuser ceph since I am not root 2024-04-25T13:23:46.807 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b[62256]: ignoring --setgroup ceph since I am not root 2024-04-25T13:23:46.807 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mds[62260]: -- 172.21.15.164:0/3802504039 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b5cd1fa40 con 0x558b5ccf2800 2024-04-25T13:23:46.807 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b[62256]: starting mds.b at 2024-04-25T13:23:46.808 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h[61931]: ignoring --setuser ceph since I am not root 2024-04-25T13:23:46.808 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h[61931]: ignoring --setgroup ceph since I am not root 2024-04-25T13:23:46.808 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mds[61935]: -- 172.21.15.164:0/1439829578 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed9a309a40 con 0x55ed9a2dd000 2024-04-25T13:23:46.808 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h[61931]: starting mds.h at 2024-04-25T13:23:46.808 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e[61620]: ignoring --setuser ceph since I am not root 2024-04-25T13:23:46.809 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e[61620]: ignoring --setgroup ceph since I am not root 2024-04-25T13:23:46.809 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mds[61624]: -- 172.21.15.164:0/829782402 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa555a7a40 con 0x55aa5557b000 2024-04-25T13:23:46.809 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e[61620]: starting mds.e at 2024-04-25T13:23:46.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/77242333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:46.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1847021135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:46.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2337848750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:46.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3146153353' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:23:46.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mon[32418]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:23:46.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mon[32418]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:23:46.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3146153353' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:23:46.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mon[32418]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:23:46.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:46 smithi164 ceph-mon[32418]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T13:23:47.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.447+0000 7f6947fff640 1 -- 172.21.15.72:0/1035740988 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f69500c1190 con 0x7f69601006e0 2024-04-25T13:23:47.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f69480a3d20 msgr2=0x7f69480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:47.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f69480a3d20 0x7f69480a61e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69600ffb50 tx=0x7f6954005f90 comp rx=0 tx=0).stop 2024-04-25T13:23:47.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 msgr2=0x7f696019e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:47.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 0x7f696019e450 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6950005940 tx=0x7f6950004270 comp rx=0 tx=0).stop 2024-04-25T13:23:47.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 shutdown_connections 2024-04-25T13:23:47.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f69480a3d20 0x7f69480a61e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:47.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6960198600 0x7f6960198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:47.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69601072f0 0x7f696019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:47.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 --2- 172.21.15.72:0/1035740988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69601006e0 0x7f696019e450 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:47.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 >> 172.21.15.72:0/1035740988 conn(0x7f69600773e0 msgr2=0x7f69601056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:47.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 shutdown_connections 2024-04-25T13:23:47.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.450+0000 7f69657ad640 1 -- 172.21.15.72:0/1035740988 wait complete. 2024-04-25T13:23:47.464 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:23:47.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 -- 172.21.15.72:0/3210957258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0102190 msgr2=0x7ff9c0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:47.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 --2- 172.21.15.72:0/3210957258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0102190 0x7ff9c0102570 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009900 tx=0x7ff9b002a410 comp rx=0 tx=0).stop 2024-04-25T13:23:47.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 -- 172.21.15.72:0/3210957258 shutdown_connections 2024-04-25T13:23:47.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 --2- 172.21.15.72:0/3210957258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c0107260 0x7ff9c010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:47.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 --2- 172.21.15.72:0/3210957258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c0102b40 0x7ff9c0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:47.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 --2- 172.21.15.72:0/3210957258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0102190 0x7ff9c0102570 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:47.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 -- 172.21.15.72:0/3210957258 >> 172.21.15.72:0/3210957258 conn(0x7ff9c00793a0 msgr2=0x7ff9c00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:47.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 -- 172.21.15.72:0/3210957258 shutdown_connections 2024-04-25T13:23:47.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 -- 172.21.15.72:0/3210957258 wait complete. 2024-04-25T13:23:47.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 Processor -- start 2024-04-25T13:23:47.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.566+0000 7ff9c827c640 1 -- start start 2024-04-25T13:23:47.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c827c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c0102190 0x7ff9c019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:47.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c827c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0107260 0x7ff9c019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:47.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c827c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c0198590 0x7ff9c0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:47.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c827c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01121d0 con 0x7ff9c0107260 2024-04-25T13:23:47.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c827c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0112050 con 0x7ff9c0102190 2024-04-25T13:23:47.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c827c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0112350 con 0x7ff9c0198590 2024-04-25T13:23:47.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c5817640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0107260 0x7ff9c019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:47.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c5817640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0107260 0x7ff9c019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43580/0 (socket says 172.21.15.72:43580) 2024-04-25T13:23:47.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c5817640 1 -- 172.21.15.72:0/2520956590 learned_addr learned my addr 172.21.15.72:0/2520956590 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:47.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c6018640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c0102190 0x7ff9c019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:47.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c5817640 1 -- 172.21.15.72:0/2520956590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c0198590 msgr2=0x7ff9c0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:47.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c5817640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c0198590 0x7ff9c0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:47.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c5817640 1 -- 172.21.15.72:0/2520956590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c0102190 msgr2=0x7ff9c019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:47.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c5817640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c0102190 0x7ff9c019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:47.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c5817640 1 -- 172.21.15.72:0/2520956590 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c01990b0 con 0x7ff9c0107260 2024-04-25T13:23:47.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c5817640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0107260 0x7ff9c019e920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc005fd0 tx=0x7ff9bc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:47.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9b77fe640 1 -- 172.21.15.72:0/2520956590 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc017070 con 0x7ff9c0107260 2024-04-25T13:23:47.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9b77fe640 1 -- 172.21.15.72:0/2520956590 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9bc009c30 con 0x7ff9c0107260 2024-04-25T13:23:47.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c01993a0 con 0x7ff9c0107260 2024-04-25T13:23:47.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9b77fe640 1 -- 172.21.15.72:0/2520956590 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc003d30 con 0x7ff9c0107260 2024-04-25T13:23:47.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.567+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c01a55f0 con 0x7ff9c0107260 2024-04-25T13:23:47.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.568+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c00691c0 con 0x7ff9c0107260 2024-04-25T13:23:47.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.569+0000 7ff9b77fe640 1 -- 172.21.15.72:0/2520956590 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff9bc003ed0 con 0x7ff9c0107260 2024-04-25T13:23:47.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.570+0000 7ff9b77fe640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7ff9ac0a3d20 0x7ff9ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:47.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.570+0000 7ff9c6018640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7ff9ac0a3d20 0x7ff9ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:47.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.570+0000 7ff9b77fe640 1 -- 172.21.15.72:0/2520956590 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8764+0+0 (secure 0 0 0) 0x7ff9bc0d05f0 con 0x7ff9c0107260 2024-04-25T13:23:47.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.571+0000 7ff9c6018640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7ff9ac0a3d20 0x7ff9ac0a61e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0031440 tx=0x7ff9b00095f0 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:47.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.575+0000 7ff9b77fe640 1 -- 172.21.15.72:0/2520956590 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9bc020030 con 0x7ff9c0107260 2024-04-25T13:23:47.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:47 smithi193 ceph-mon[32743]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:47.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:47 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1035740988' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:23:47.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:47.753+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff9c0102b40 con 0x7ff9c0107260 2024-04-25T13:23:47.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:47 smithi072 ceph-mon[29199]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:47.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:47 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1035740988' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:23:47.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:47 smithi164 ceph-mon[32418]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:23:47.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:47 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1035740988' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:23:48.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.459+0000 7ff9b77fe640 1 -- 172.21.15.72:0/2520956590 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7ff9bc009de0 con 0x7ff9c0107260 2024-04-25T13:23:48.460 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:23:48.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.462+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7ff9ac0a3d20 msgr2=0x7ff9ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.462+0000 7ff9c827c640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7ff9ac0a3d20 0x7ff9ac0a61e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0031440 tx=0x7ff9b00095f0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.462+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0107260 msgr2=0x7ff9c019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.462+0000 7ff9c827c640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0107260 0x7ff9c019e920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc005fd0 tx=0x7ff9bc00dde0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.463+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 shutdown_connections 2024-04-25T13:23:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.463+0000 7ff9c827c640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7ff9ac0a3d20 0x7ff9ac0a61e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.463+0000 7ff9c827c640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c0198590 0x7ff9c0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.463+0000 7ff9c827c640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9c0107260 0x7ff9c019e920 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.463+0000 7ff9c827c640 1 --2- 172.21.15.72:0/2520956590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c0102190 0x7ff9c019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.463+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 >> 172.21.15.72:0/2520956590 conn(0x7ff9c00793a0 msgr2=0x7ff9c00fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:48.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.463+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 shutdown_connections 2024-04-25T13:23:48.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.463+0000 7ff9c827c640 1 -- 172.21.15.72:0/2520956590 wait complete. 2024-04-25T13:23:48.476 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:23:48.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.578+0000 7f665f8d0640 1 -- 172.21.15.72:0/26547725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 msgr2=0x7f665810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.578+0000 7f665f8d0640 1 --2- 172.21.15.72:0/26547725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 0x7f665810dd30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6650009ca0 tx=0x7f665002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.578+0000 7f665f8d0640 1 -- 172.21.15.72:0/26547725 shutdown_connections 2024-04-25T13:23:48.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.578+0000 7f665f8d0640 1 --2- 172.21.15.72:0/26547725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 0x7f665810dd30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.578+0000 7f665f8d0640 1 --2- 172.21.15.72:0/26547725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658101a40 0x7f6658105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.578+0000 7f665f8d0640 1 --2- 172.21.15.72:0/26547725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658101090 0x7f6658101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.578+0000 7f665f8d0640 1 -- 172.21.15.72:0/26547725 >> 172.21.15.72:0/26547725 conn(0x7f6658077f50 msgr2=0x7f6658078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 -- 172.21.15.72:0/26547725 shutdown_connections 2024-04-25T13:23:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 -- 172.21.15.72:0/26547725 wait complete. 2024-04-25T13:23:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 Processor -- start 2024-04-25T13:23:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 -- start start 2024-04-25T13:23:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658101090 0x7f6658198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 0x7f6658198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f665819cfc0 0x7f665819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:48.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f66581121e0 con 0x7f6658106160 2024-04-25T13:23:48.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6658112060 con 0x7f665819cfc0 2024-04-25T13:23:48.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665f8d0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6658112360 con 0x7f6658101090 2024-04-25T13:23:48.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665ce6b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 0x7f6658198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:48.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665ce6b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 0x7f6658198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43584/0 (socket says 172.21.15.72:43584) 2024-04-25T13:23:48.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.579+0000 7f665ce6b640 1 -- 172.21.15.72:0/3273073312 learned_addr learned my addr 172.21.15.72:0/3273073312 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-25T13:23:48.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665de6d640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f665819cfc0 0x7f665819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:48.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665d66c640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658101090 0x7f6658198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:48.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665ce6b640 1 -- 172.21.15.72:0/3273073312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658101090 msgr2=0x7f6658198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665ce6b640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658101090 0x7f6658198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665ce6b640 1 -- 172.21.15.72:0/3273073312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f665819cfc0 msgr2=0x7f665819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665ce6b640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f665819cfc0 0x7f665819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665ce6b640 1 -- 172.21.15.72:0/3273073312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665819dab0 con 0x7f6658106160 2024-04-25T13:23:48.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665ce6b640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 0x7f6658198c40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6648005fd0 tx=0x7f664800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:48.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f664e7fc640 1 -- 172.21.15.72:0/3273073312 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648017070 con 0x7f6658106160 2024-04-25T13:23:48.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f664e7fc640 1 -- 172.21.15.72:0/3273073312 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6648009c30 con 0x7f6658106160 2024-04-25T13:23:48.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665819dda0 con 0x7f6658106160 2024-04-25T13:23:48.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f664e7fc640 1 -- 172.21.15.72:0/3273073312 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648003d30 con 0x7f6658106160 2024-04-25T13:23:48.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.580+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66581a55e0 con 0x7f6658106160 2024-04-25T13:23:48.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.581+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6658068d70 con 0x7f6658106160 2024-04-25T13:23:48.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.582+0000 7f664e7fc640 1 -- 172.21.15.72:0/3273073312 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6648003ed0 con 0x7f6658106160 2024-04-25T13:23:48.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.583+0000 7f664e7fc640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f66440a3da0 0x7f66440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:48.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.583+0000 7f665d66c640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f66440a3da0 0x7f66440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:48.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.583+0000 7f664e7fc640 1 -- 172.21.15.72:0/3273073312 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8173+0+0 (secure 0 0 0) 0x7f66480d01e0 con 0x7f6658106160 2024-04-25T13:23:48.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.583+0000 7f665d66c640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f66440a3da0 0x7f66440a6260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6654005fd0 tx=0x7f6654005850 comp rx=0 tx=0).ready entity=mgr.15276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:48.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.587+0000 7f664e7fc640 1 -- 172.21.15.72:0/3273073312 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6648020030 con 0x7f6658106160 2024-04-25T13:23:48.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:23:48.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:23:48.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: Cluster is now healthy 2024-04-25T13:23:48.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/1035740988' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:23:48.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.193:6834/478797132,v1:172.21.15.193:6835/478797132] up:boot 2024-04-25T13:23:48.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.164:6834/1891715676,v1:172.21.15.164:6835/1891715676] up:boot 2024-04-25T13:23:48.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.193:6836/4003319350,v1:172.21.15.193:6838/4003319350] up:boot 2024-04-25T13:23:48.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.193:6837/4222140390,v1:172.21.15.193:6839/4222140390] up:boot 2024-04-25T13:23:48.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.164:6836/3606522871,v1:172.21.15.164:6837/3606522871] up:boot 2024-04-25T13:23:48.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: mds.? [v2:172.21.15.164:6838/3251481163,v1:172.21.15.164:6839/3251481163] up:boot 2024-04-25T13:23:48.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: fsmap 12 up:standby 2024-04-25T13:23:48.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:23:48.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:23:48.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:23:48.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:23:48.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:23:48.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:23:48.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:48 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2520956590' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:23:48.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:48.773+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6658101a40 con 0x7f6658106160 2024-04-25T13:23:48.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:23:48.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:23:48.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: Cluster is now healthy 2024-04-25T13:23:48.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/1035740988' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:23:48.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.193:6834/478797132,v1:172.21.15.193:6835/478797132] up:boot 2024-04-25T13:23:48.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.164:6834/1891715676,v1:172.21.15.164:6835/1891715676] up:boot 2024-04-25T13:23:48.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.193:6836/4003319350,v1:172.21.15.193:6838/4003319350] up:boot 2024-04-25T13:23:48.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.193:6837/4222140390,v1:172.21.15.193:6839/4222140390] up:boot 2024-04-25T13:23:48.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.164:6836/3606522871,v1:172.21.15.164:6837/3606522871] up:boot 2024-04-25T13:23:48.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: mds.? [v2:172.21.15.164:6838/3251481163,v1:172.21.15.164:6839/3251481163] up:boot 2024-04-25T13:23:48.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: fsmap 12 up:standby 2024-04-25T13:23:48.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:23:48.775 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:23:48.776 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:23:48.776 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:23:48.776 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:23:48.776 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:23:48.776 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:48 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2520956590' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:23:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:23:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:23:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: Cluster is now healthy 2024-04-25T13:23:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/1035740988' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:23:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.193:6834/478797132,v1:172.21.15.193:6835/478797132] up:boot 2024-04-25T13:23:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.164:6834/1891715676,v1:172.21.15.164:6835/1891715676] up:boot 2024-04-25T13:23:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.193:6836/4003319350,v1:172.21.15.193:6838/4003319350] up:boot 2024-04-25T13:23:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.193:6837/4222140390,v1:172.21.15.193:6839/4222140390] up:boot 2024-04-25T13:23:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.164:6836/3606522871,v1:172.21.15.164:6837/3606522871] up:boot 2024-04-25T13:23:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: mds.? [v2:172.21.15.164:6838/3251481163,v1:172.21.15.164:6839/3251481163] up:boot 2024-04-25T13:23:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: fsmap 12 up:standby 2024-04-25T13:23:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:23:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:23:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:23:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:23:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:23:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: from='mgr.15276 172.21.15.72:0/3742555672' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:23:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:48 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2520956590' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:23:49.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.462+0000 7f664e7fc640 1 -- 172.21.15.72:0/3273073312 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f664809ca30 con 0x7f6658106160 2024-04-25T13:23:49.463 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' removed 2024-04-25T13:23:49.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f66440a3da0 msgr2=0x7f66440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:49.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f66440a3da0 0x7f66440a6260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6654005fd0 tx=0x7f6654005850 comp rx=0 tx=0).stop 2024-04-25T13:23:49.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 msgr2=0x7f6658198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:49.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 0x7f6658198c40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6648005fd0 tx=0x7f664800dde0 comp rx=0 tx=0).stop 2024-04-25T13:23:49.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 shutdown_connections 2024-04-25T13:23:49.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:6800/3170992219,v1:172.21.15.72:6801/3170992219] conn(0x7f66440a3da0 0x7f66440a6260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:49.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f665819cfc0 0x7f665819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:49.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6658106160 0x7f6658198c40 secure :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6648005fd0 tx=0x7f664800dde0 comp rx=0 tx=0).stop 2024-04-25T13:23:49.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 --2- 172.21.15.72:0/3273073312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658101090 0x7f6658198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:49.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.466+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 >> 172.21.15.72:0/3273073312 conn(0x7f6658077f50 msgr2=0x7f66580ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:49.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.467+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 shutdown_connections 2024-04-25T13:23:49.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-25T13:23:49.467+0000 7f665f8d0640 1 -- 172.21.15.72:0/3273073312 wait complete. 2024-04-25T13:23:49.482 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:23:49.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:23:49.540 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:23:49.572 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T13:23:49.573 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:23:49.589 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:23:49.605 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:23:49.630 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T13:23:49.630 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T13:23:49.631 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:23:49.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:49 smithi193 ceph-mon[32743]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:23:49.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:49 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/2520956590' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:23:49.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:49 smithi193 ceph-mon[32743]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:23:49.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:49 smithi193 ceph-mon[32743]: from='client.? 172.21.15.72:0/3273073312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:23:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:49 smithi072 ceph-mon[29199]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:23:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:49 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/2520956590' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:23:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:49 smithi072 ceph-mon[29199]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:23:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:49 smithi072 ceph-mon[29199]: from='client.? 172.21.15.72:0/3273073312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:23:49.774 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:49 smithi072 systemd[1]: Stopping Ceph mon.a for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:23:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:49 smithi164 ceph-mon[32418]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:23:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:49 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/2520956590' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:23:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:49 smithi164 ceph-mon[32418]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:23:49.807 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:49 smithi164 ceph-mon[32418]: from='client.? 172.21.15.72:0/3273073312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:23:50.223 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:49 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a[29195]: 2024-04-25T13:23:49.937+0000 7f735d399700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:23:50.223 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:49 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a[29195]: 2024-04-25T13:23:49.937+0000 7f735d399700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T13:23:50.223 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:49 smithi072 podman[87075]: 2024-04-25 13:23:49.959238987 +0000 UTC m=+0.144701751 container died f54ab9bf98cbd2517159d9806b4f360e88cf32bd041be04d0ff8b0fdd263ba46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:23:50.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:50 smithi072 podman[87075]: 2024-04-25 13:23:50.325207103 +0000 UTC m=+0.510669904 container cleanup f54ab9bf98cbd2517159d9806b4f360e88cf32bd041be04d0ff8b0fdd263ba46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T13:23:50.524 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 25 13:23:50 smithi072 bash[87075]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:23:50.751 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T13:23:50.796 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:23:50.796 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T13:23:50.796 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T13:23:50.797 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.b 2024-04-25T13:23:51.095 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:50 smithi164 systemd[1]: Stopping Ceph mon.b for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:23:51.419 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-b[32414]: 2024-04-25T13:23:51.092+0000 7ff2d92d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:23:51.420 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-b[32414]: 2024-04-25T13:23:51.092+0000 7ff2d92d5700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:23:51.420 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:51 smithi164 podman[67617]: 2024-04-25 13:23:51.196296755 +0000 UTC m=+0.208158629 container died a54523d0c0ee20c244e733fe05eaa7bd70aa6b91b8bf1fde20f173e238d1f479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:23:51.720 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:51 smithi164 podman[67617]: 2024-04-25 13:23:51.536888859 +0000 UTC m=+0.548750740 container cleanup a54523d0c0ee20c244e733fe05eaa7bd70aa6b91b8bf1fde20f173e238d1f479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T13:23:51.720 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:51 smithi164 bash[67617]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-b 2024-04-25T13:23:51.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T13:23:51.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:51 smithi164 podman[67631]: 2024-04-25 13:23:51.720402514 +0000 UTC m=+0.527918648 container remove a54523d0c0ee20c244e733fe05eaa7bd70aa6b91b8bf1fde20f173e238d1f479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T13:23:51.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:51 smithi164 systemd[1]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.b.service: Deactivated successfully. 2024-04-25T13:23:51.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:51 smithi164 systemd[1]: Stopped Ceph mon.b for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:23:51.975 INFO:journalctl@ceph.mon.b.smithi164.stdout:Apr 25 13:23:51 smithi164 systemd[1]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.b.service: Consumed 10.901s CPU time. 2024-04-25T13:23:51.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:23:51.999 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T13:23:51.999 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T13:23:51.999 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.c 2024-04-25T13:23:52.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:52 smithi193 systemd[1]: Stopping Ceph mon.c for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:23:52.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:52 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-c[32727]: 2024-04-25T13:23:52.302+0000 7f59d90a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:23:52.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:52 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-c[32727]: 2024-04-25T13:23:52.302+0000 7f59d90a6700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:23:52.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:52 smithi193 podman[68559]: 2024-04-25 13:23:52.517319347 +0000 UTC m=+0.321606951 container died 6e1c06de1eb50c9e1ba8c5f04ae790b57a3c2a01b7c433a0b4977f1b8d7ea19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:23:53.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:52 smithi193 podman[68559]: 2024-04-25 13:23:52.886410125 +0000 UTC m=+0.690697736 container cleanup 6e1c06de1eb50c9e1ba8c5f04ae790b57a3c2a01b7c433a0b4977f1b8d7ea19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:23:53.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Apr 25 13:23:52 smithi193 bash[68559]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mon-c 2024-04-25T13:23:53.304 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T13:23:53.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:23:53.355 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T13:23:53.355 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T13:23:53.356 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:23:53.709 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:23:53 smithi072 systemd[1]: Stopping Ceph mgr.x for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:23:53.969 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:23:53 smithi072 podman[87198]: 2024-04-25 13:23:53.712718758 +0000 UTC m=+0.161794081 container died fafea5b7b7d7652c4dbd1f2c214bd60f6fa5dce10361d6ef8f121bac15ade058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T13:23:54.274 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:23:54 smithi072 podman[87198]: 2024-04-25 13:23:54.086360955 +0000 UTC m=+0.535436277 container cleanup fafea5b7b7d7652c4dbd1f2c214bd60f6fa5dce10361d6ef8f121bac15ade058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:23:54.274 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 25 13:23:54 smithi072 bash[87198]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-x 2024-04-25T13:23:54.507 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T13:23:54.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:23:54.541 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T13:23:54.541 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T13:23:54.542 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.y 2024-04-25T13:23:54.877 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:23:54 smithi164 systemd[1]: Stopping Ceph mgr.y for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:23:55.157 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:23:54 smithi164 podman[67738]: 2024-04-25 13:23:54.877559929 +0000 UTC m=+0.146695277 container died 69b024243b70d41890028e011604025395142883ba403ac686e84d067dd869ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:23:55.525 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:23:55 smithi164 podman[67738]: 2024-04-25 13:23:55.273757888 +0000 UTC m=+0.542893228 container cleanup 69b024243b70d41890028e011604025395142883ba403ac686e84d067dd869ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T13:23:55.525 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Apr 25 13:23:55 smithi164 bash[67738]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-y 2024-04-25T13:23:55.703 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T13:23:55.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:23:55.736 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T13:23:55.736 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T13:23:55.737 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.z 2024-04-25T13:23:56.036 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:23:55 smithi193 systemd[1]: Stopping Ceph mgr.z for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:23:56.323 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:23:56 smithi193 podman[68681]: 2024-04-25 13:23:56.036727993 +0000 UTC m=+0.104344051 container died d1e176b2ff2e9327e63ce62734327e23b46cbf4cf23736713464e32be7f4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:23:56.583 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:23:56 smithi193 podman[68681]: 2024-04-25 13:23:56.324009141 +0000 UTC m=+0.391625200 container cleanup d1e176b2ff2e9327e63ce62734327e23b46cbf4cf23736713464e32be7f4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T13:23:56.583 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Apr 25 13:23:56 smithi193 bash[68681]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mgr-z 2024-04-25T13:23:56.753 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T13:23:56.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:23:56.787 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T13:23:56.787 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T13:23:56.787 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.0 2024-04-25T13:23:57.084 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:23:56 smithi072 systemd[1]: Stopping Ceph osd.0 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:23:57.524 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:23:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0[44263]: 2024-04-25T13:23:57.083+0000 7fd9db1e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:23:57.524 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:23:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0[44263]: 2024-04-25T13:23:57.083+0000 7fd9db1e0700 -1 osd.0 102 *** Got signal Terminated *** 2024-04-25T13:23:57.524 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:23:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0[44263]: 2024-04-25T13:23:57.083+0000 7fd9db1e0700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:24:02.774 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:02 smithi072 podman[87321]: 2024-04-25 13:24:02.511801607 +0000 UTC m=+5.533429884 container died 3be2dc4280518ade35d3035ebbff35d8c0be5500885ea216c996dba2d69f017d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:24:03.132 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:02 smithi072 podman[87321]: 2024-04-25 13:24:02.863415558 +0000 UTC m=+5.885043817 container cleanup 3be2dc4280518ade35d3035ebbff35d8c0be5500885ea216c996dba2d69f017d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:24:03.132 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:02 smithi072 bash[87321]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0 2024-04-25T13:24:03.479 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:03 smithi072 podman[87341]: 2024-04-25 13:24:03.13247521 +0000 UTC m=+0.615237581 container remove 3be2dc4280518ade35d3035ebbff35d8c0be5500885ea216c996dba2d69f017d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:24:03.774 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:03 smithi072 podman[87432]: 2024-04-25 13:24:03.38181294 +0000 UTC m=+0.018034764 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:03.775 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:03 smithi072 podman[87432]: 2024-04-25 13:24:03.485867586 +0000 UTC m=+0.122089442 container create 16c4dfd30f4d78c342a6535eef6ae4f882573289dfb47e6b77aaa89cef234723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T13:24:03.775 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:03 smithi072 podman[87432]: 2024-04-25 13:24:03.723521003 +0000 UTC m=+0.359742827 container init 16c4dfd30f4d78c342a6535eef6ae4f882573289dfb47e6b77aaa89cef234723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:24:03.775 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:03 smithi072 podman[87432]: 2024-04-25 13:24:03.727431133 +0000 UTC m=+0.363652950 container start 16c4dfd30f4d78c342a6535eef6ae4f882573289dfb47e6b77aaa89cef234723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T13:24:04.026 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:03 smithi072 podman[87432]: 2024-04-25 13:24:03.815276843 +0000 UTC m=+0.451498670 container attach 16c4dfd30f4d78c342a6535eef6ae4f882573289dfb47e6b77aaa89cef234723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:24:04.027 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 25 13:24:04 smithi072 podman[87432]: 2024-04-25 13:24:04.026653595 +0000 UTC m=+0.662875414 container died 16c4dfd30f4d78c342a6535eef6ae4f882573289dfb47e6b77aaa89cef234723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T13:24:04.615 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T13:24:04.654 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:24:04.654 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T13:24:04.654 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T13:24:04.654 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.1 2024-04-25T13:24:04.986 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:04 smithi164 systemd[1]: Stopping Ceph osd.1 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:24:05.307 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1[37057]: 2024-04-25T13:24:04.985+0000 7f3a24921700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:24:05.307 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1[37057]: 2024-04-25T13:24:04.985+0000 7f3a24921700 -1 osd.1 102 *** Got signal Terminated *** 2024-04-25T13:24:05.307 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1[37057]: 2024-04-25T13:24:04.985+0000 7f3a24921700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:24:10.637 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:10 smithi164 podman[67873]: 2024-04-25 13:24:10.364512851 +0000 UTC m=+5.506747568 container died 5153eb3fcb9bbe77a02a2330f0a2545edc26c1c4ed82485d7d07822ad0016f03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T13:24:11.057 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:10 smithi164 podman[67873]: 2024-04-25 13:24:10.702597386 +0000 UTC m=+5.844832110 container cleanup 5153eb3fcb9bbe77a02a2330f0a2545edc26c1c4ed82485d7d07822ad0016f03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T13:24:11.057 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:10 smithi164 bash[67873]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1 2024-04-25T13:24:11.057 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:10 smithi164 podman[67891]: 2024-04-25 13:24:10.887016454 +0000 UTC m=+0.516461178 container remove 5153eb3fcb9bbe77a02a2330f0a2545edc26c1c4ed82485d7d07822ad0016f03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:24:11.616 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:11 smithi164 podman[67985]: 2024-04-25 13:24:11.226986166 +0000 UTC m=+0.018466146 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:11.616 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:11 smithi164 podman[67985]: 2024-04-25 13:24:11.363099874 +0000 UTC m=+0.154579848 container create f6f9beeaf701f8f88738323b8e8df240af40f6c46d9a22ec83c8ea8c5eeb4773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T13:24:11.911 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:11 smithi164 podman[67985]: 2024-04-25 13:24:11.616825057 +0000 UTC m=+0.408305035 container init f6f9beeaf701f8f88738323b8e8df240af40f6c46d9a22ec83c8ea8c5eeb4773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:24:11.911 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:11 smithi164 podman[67985]: 2024-04-25 13:24:11.620701967 +0000 UTC m=+0.412181938 container start f6f9beeaf701f8f88738323b8e8df240af40f6c46d9a22ec83c8ea8c5eeb4773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T13:24:11.912 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:11 smithi164 podman[67985]: 2024-04-25 13:24:11.699961146 +0000 UTC m=+0.491441120 container attach f6f9beeaf701f8f88738323b8e8df240af40f6c46d9a22ec83c8ea8c5eeb4773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T13:24:12.252 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:11 smithi164 podman[67985]: 2024-04-25 13:24:11.912174591 +0000 UTC m=+0.703654555 container died f6f9beeaf701f8f88738323b8e8df240af40f6c46d9a22ec83c8ea8c5eeb4773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:24:12.504 INFO:journalctl@ceph.osd.1.smithi164.stdout:Apr 25 13:24:12 smithi164 podman[68012]: 2024-04-25 13:24:12.4872132 +0000 UTC m=+0.569230523 container remove f6f9beeaf701f8f88738323b8e8df240af40f6c46d9a22ec83c8ea8c5eeb4773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T13:24:12.510 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T13:24:12.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:24:12.547 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T13:24:12.547 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T13:24:12.547 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.2 2024-04-25T13:24:12.844 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:12 smithi193 systemd[1]: Stopping Ceph osd.2 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:24:13.232 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:12 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2[37928]: 2024-04-25T13:24:12.843+0000 7f84a83c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:24:13.232 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:12 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2[37928]: 2024-04-25T13:24:12.843+0000 7f84a83c9700 -1 osd.2 102 *** Got signal Terminated *** 2024-04-25T13:24:13.232 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:12 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2[37928]: 2024-04-25T13:24:12.843+0000 7f84a83c9700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:24:18.560 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:18 smithi193 podman[68821]: 2024-04-25 13:24:18.280348239 +0000 UTC m=+5.534196354 container died f66947b8701a03bd065c2634c093e7eb7dd61eb5053d479924299bdb72cc3345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T13:24:18.561 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:18 smithi193 podman[68821]: 2024-04-25 13:24:18.56082908 +0000 UTC m=+5.814677205 container cleanup f66947b8701a03bd065c2634c093e7eb7dd61eb5053d479924299bdb72cc3345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T13:24:18.811 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:18 smithi193 bash[68821]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2 2024-04-25T13:24:19.116 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:18 smithi193 podman[68839]: 2024-04-25 13:24:18.811970314 +0000 UTC m=+0.531264733 container remove f66947b8701a03bd065c2634c093e7eb7dd61eb5053d479924299bdb72cc3345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T13:24:19.373 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:19 smithi193 podman[68931]: 2024-04-25 13:24:19.020126821 +0000 UTC m=+0.018364941 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:19.373 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:19 smithi193 podman[68931]: 2024-04-25 13:24:19.146155502 +0000 UTC m=+0.144393608 container create d95f953a88d423e5e49f19eb22705bda4857faa12a14e08cd450931eaed57014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T13:24:19.373 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:19 smithi193 podman[68931]: 2024-04-25 13:24:19.373585788 +0000 UTC m=+0.371823894 container init d95f953a88d423e5e49f19eb22705bda4857faa12a14e08cd450931eaed57014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T13:24:19.679 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:19 smithi193 podman[68931]: 2024-04-25 13:24:19.376726791 +0000 UTC m=+0.374964900 container start d95f953a88d423e5e49f19eb22705bda4857faa12a14e08cd450931eaed57014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T13:24:19.679 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:19 smithi193 podman[68931]: 2024-04-25 13:24:19.457245465 +0000 UTC m=+0.455483578 container attach d95f953a88d423e5e49f19eb22705bda4857faa12a14e08cd450931eaed57014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T13:24:19.981 INFO:journalctl@ceph.osd.2.smithi193.stdout:Apr 25 13:24:19 smithi193 podman[68931]: 2024-04-25 13:24:19.679664619 +0000 UTC m=+0.677902720 container died d95f953a88d423e5e49f19eb22705bda4857faa12a14e08cd450931eaed57014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:24:20.275 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T13:24:20.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:24:20.313 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T13:24:20.313 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T13:24:20.314 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.3 2024-04-25T13:24:20.612 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:20 smithi072 systemd[1]: Stopping Ceph osd.3 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:24:21.024 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:20 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3[51236]: 2024-04-25T13:24:20.611+0000 7f969abe7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:24:21.024 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:20 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3[51236]: 2024-04-25T13:24:20.611+0000 7f969abe7700 -1 osd.3 102 *** Got signal Terminated *** 2024-04-25T13:24:21.024 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:20 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3[51236]: 2024-04-25T13:24:20.611+0000 7f969abe7700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:24:24.232 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:23.969+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:24.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:23 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:23.875+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:25.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:24 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:24.716+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:25.025 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:24 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:24.871+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:25.231 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:24.923+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:25.524 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:25 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3[51236]: 2024-04-25T13:24:25.188+0000 7f96971db700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.770829+0000 front 2024-04-25T13:23:58.770890+0000 (oldest deadline 2024-04-25T13:24:25.171577+0000) 2024-04-25T13:24:26.025 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:25 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:25.868+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:26.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:25 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:25.740+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:26.231 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:25 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:25.970+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:26.367 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:26 smithi072 podman[87552]: 2024-04-25 13:24:26.098206228 +0000 UTC m=+5.592423315 container died fc1c2fc3b01dda4da5da9c73d846185edf2dd161fc0c389f877f89b4d538b8d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:24:26.735 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:26 smithi072 podman[87552]: 2024-04-25 13:24:26.465981155 +0000 UTC m=+5.960198243 container cleanup fc1c2fc3b01dda4da5da9c73d846185edf2dd161fc0c389f877f89b4d538b8d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T13:24:26.736 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:26 smithi072 bash[87552]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3 2024-04-25T13:24:27.024 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:26 smithi072 podman[87569]: 2024-04-25 13:24:26.735074334 +0000 UTC m=+0.639752371 container remove fc1c2fc3b01dda4da5da9c73d846185edf2dd161fc0c389f877f89b4d538b8d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:24:27.025 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:26 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:26.893+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:27.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:26 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:26.744+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:27.231 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:27.018+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:27.232 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:26 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:26.780+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:27.351 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:27 smithi072 podman[87658]: 2024-04-25 13:24:26.991074696 +0000 UTC m=+0.017962593 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:27.351 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:27 smithi072 podman[87658]: 2024-04-25 13:24:27.122134254 +0000 UTC m=+0.149022176 container create abb347228077ea0fe6c7d3f1b9c07b77c4f61ac69b36ea884b0e55bd3df8b592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T13:24:27.351 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:27 smithi072 podman[87658]: 2024-04-25 13:24:27.351047749 +0000 UTC m=+0.377935629 container init abb347228077ea0fe6c7d3f1b9c07b77c4f61ac69b36ea884b0e55bd3df8b592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:24:27.648 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:27 smithi072 podman[87658]: 2024-04-25 13:24:27.354297418 +0000 UTC m=+0.381185299 container start abb347228077ea0fe6c7d3f1b9c07b77c4f61ac69b36ea884b0e55bd3df8b592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T13:24:27.649 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:27 smithi072 podman[87658]: 2024-04-25 13:24:27.434698295 +0000 UTC m=+0.461586205 container attach abb347228077ea0fe6c7d3f1b9c07b77c4f61ac69b36ea884b0e55bd3df8b592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:24:27.731 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:27.292+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:27.935 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:27 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:27.743+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:27.935 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 25 13:24:27 smithi072 podman[87658]: 2024-04-25 13:24:27.649175688 +0000 UTC m=+0.676063565 container died abb347228077ea0fe6c7d3f1b9c07b77c4f61ac69b36ea884b0e55bd3df8b592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:24:28.035 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:27.771+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:28.209 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:27 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:27.934+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:28.232 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T13:24:28.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:24:28.269 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T13:24:28.269 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T13:24:28.269 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.4 2024-04-25T13:24:28.294 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:28 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:28.023+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:28.304 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:28.034+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:28.557 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:28 smithi164 systemd[1]: Stopping Ceph osd.4 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:24:28.731 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:28.303+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:29.023 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:28 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4[43693]: 2024-04-25T13:24:28.603+0000 7f9c02653700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:24:29.024 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:28 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4[43693]: 2024-04-25T13:24:28.603+0000 7f9c02653700 -1 osd.4 102 *** Got signal Terminated *** 2024-04-25T13:24:29.024 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:28 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4[43693]: 2024-04-25T13:24:28.603+0000 7f9c02653700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:24:29.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:28 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:28.891+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:29.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:28 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:28.731+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:29.077 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:28.760+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:29.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:29 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:29.022+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:29.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:29 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:29.077+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:29.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:29 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:29.259+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:29.684 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:29 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:29.684+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:29.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:29 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:29.728+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:30.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:29 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:29.927+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:30.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:29 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:29.984+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:30.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:30 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:30.052+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:30.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:30 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:30.294+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:30.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:30 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:30.702+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:31.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:30 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:30.939+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:31.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:30 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:30.702+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:31.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:30 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:30.981+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:31.328 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:31 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:31.066+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:31.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:31 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:31.328+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:31.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:31 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:31.328+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:31.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:31 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:31.667+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:32.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:31 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:31.722+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:32.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:31 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:31.930+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:32.025 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:31 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:31.930+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:32.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:32 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:32.000+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:32.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:32 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:32.000+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:32.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:32 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:32.097+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:32.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:32 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:32.097+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:32.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:32 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:32.346+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:32.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:32 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:32.346+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:32.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:32 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:32.694+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:32.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:32 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:32.694+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.511952+0000 front 2024-04-25T13:24:06.512005+0000 (oldest deadline 2024-04-25T13:24:31.812001+0000) 2024-04-25T13:24:33.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:32 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:32.721+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:33.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:32 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:32.917+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:33.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:32 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:32.917+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:33.208 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:32 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:32.958+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:33.209 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:32 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:32.958+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:33.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:33 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:33.124+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:33.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:33 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:33.124+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:33.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:33 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:33.355+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:33.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:33 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:33.355+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:33.948 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:33 smithi164 podman[68090]: 2024-04-25 13:24:33.930796116 +0000 UTC m=+5.465371312 container died a96f148cd4f06eb9353263e5de433e82cae7e65adcb6f6f51dfa41eda17c5949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:24:33.949 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:33 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:33.946+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:33.949 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:33 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:33.946+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:33.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:33 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:33.687+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:33.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:33 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:33.687+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.511952+0000 front 2024-04-25T13:24:06.512005+0000 (oldest deadline 2024-04-25T13:24:31.812001+0000) 2024-04-25T13:24:34.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:33 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:33.752+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:34.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:33 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:33.752+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:34.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:33 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:33.877+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:34.025 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:33 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:33.877+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:34.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:34 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:34.161+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:34.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:34 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:34.161+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:34.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:34 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:34.325+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:34.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:34 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:34.325+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:34.557 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:34 smithi164 podman[68090]: 2024-04-25 13:24:34.240058398 +0000 UTC m=+5.774633584 container cleanup a96f148cd4f06eb9353263e5de433e82cae7e65adcb6f6f51dfa41eda17c5949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:24:34.557 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:34 smithi164 bash[68090]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4 2024-04-25T13:24:34.557 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:34 smithi164 podman[68118]: 2024-04-25 13:24:34.416160658 +0000 UTC m=+0.484832663 container remove a96f148cd4f06eb9353263e5de433e82cae7e65adcb6f6f51dfa41eda17c5949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T13:24:34.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:34 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:34.722+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:34.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:34 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:34.722+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.511952+0000 front 2024-04-25T13:24:06.512005+0000 (oldest deadline 2024-04-25T13:24:31.812001+0000) 2024-04-25T13:24:35.011 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:34 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:34.982+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:35.011 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:34 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:34.982+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:35.012 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:34 smithi164 podman[68212]: 2024-04-25 13:24:34.647503417 +0000 UTC m=+0.018106220 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:35.012 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:34 smithi164 podman[68212]: 2024-04-25 13:24:34.750296636 +0000 UTC m=+0.120899423 container create ae4e8dfa80f57406d3d286da3b3d5c3fb798685468ce6eb46003d95ae1b15f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T13:24:35.012 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:34 smithi164 podman[68212]: 2024-04-25 13:24:34.92849115 +0000 UTC m=+0.299093937 container init ae4e8dfa80f57406d3d286da3b3d5c3fb798685468ce6eb46003d95ae1b15f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:24:35.012 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:34 smithi164 podman[68212]: 2024-04-25 13:24:34.93156575 +0000 UTC m=+0.302168537 container start ae4e8dfa80f57406d3d286da3b3d5c3fb798685468ce6eb46003d95ae1b15f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:24:35.012 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:35 smithi164 podman[68212]: 2024-04-25 13:24:35.011920295 +0000 UTC m=+0.382523085 container attach ae4e8dfa80f57406d3d286da3b3d5c3fb798685468ce6eb46003d95ae1b15f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T13:24:35.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:34 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:34.865+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:35.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:34 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:34.865+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:35.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:34 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:34.747+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:35.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:34 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:34.747+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:35.307 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:35 smithi164 podman[68212]: 2024-04-25 13:24:35.235086171 +0000 UTC m=+0.605688958 container died ae4e8dfa80f57406d3d286da3b3d5c3fb798685468ce6eb46003d95ae1b15f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:24:35.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:35 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:35.174+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:35.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:35 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:35.174+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:35.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:35 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:35.331+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:35.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:35 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:35.331+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:35.782 INFO:journalctl@ceph.osd.4.smithi164.stdout:Apr 25 13:24:35 smithi164 podman[68238]: 2024-04-25 13:24:35.759187256 +0000 UTC m=+0.518284415 container remove ae4e8dfa80f57406d3d286da3b3d5c3fb798685468ce6eb46003d95ae1b15f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:24:35.787 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T13:24:35.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:24:35.867 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T13:24:35.867 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T13:24:35.867 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.5 2024-04-25T13:24:35.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:35 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:35.701+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:35.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:35 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:35.701+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.511952+0000 front 2024-04-25T13:24:06.512005+0000 (oldest deadline 2024-04-25T13:24:31.812001+0000) 2024-04-25T13:24:35.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:35 smithi193 systemd[1]: Stopping Ceph osd.5 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:24:36.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:35 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:35.884+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:36.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:35 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:35.884+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:36.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:35 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:35.728+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:36.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:35 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:35.728+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:36.057 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:35 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:35.973+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:36.057 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:36 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:36.012+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:36.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:36 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:36.012+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:36.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:36 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:36.222+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:36.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:36 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:36.222+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:36.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:36 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:36.323+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:36.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:36 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:36.323+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:36.483 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:36 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:36.153+0000 7f79ecd7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:24:36.483 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:36 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:36.153+0000 7f79ecd7e700 -1 osd.5 102 *** Got signal Terminated *** 2024-04-25T13:24:36.483 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:36 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:36.153+0000 7f79ecd7e700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:24:36.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:36 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:36.671+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:36.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:36 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:36.671+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.511952+0000 front 2024-04-25T13:24:06.512005+0000 (oldest deadline 2024-04-25T13:24:31.812001+0000) 2024-04-25T13:24:37.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:36 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:36.854+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:37.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:36 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:36.854+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:37.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:36 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:36.719+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:37.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:36 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:36.719+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:37.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:36 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:36.951+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:37.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:37 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:37.059+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:37.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:37 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:37.059+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:37.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:37 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:37.175+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:37.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:37 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:37.175+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:37.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:37 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:37.365+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:37.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:37 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:37.365+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:37.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:37 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:37.632+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:37.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:37 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:37.632+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.511952+0000 front 2024-04-25T13:24:06.512005+0000 (oldest deadline 2024-04-25T13:24:31.812001+0000) 2024-04-25T13:24:38.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:37 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:37.868+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:38.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:37 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:37.868+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:38.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:37 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:37.713+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:38.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:37 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:37.713+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:38.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:37 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:37.929+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:38.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:38 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:38.073+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:38.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:38 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:38.073+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:38.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:38 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:38.134+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:38.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:38 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:38.134+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:38.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:38 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:38.348+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:38.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:38 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:38.348+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:38.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:38 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:38.667+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:38.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:38 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:38.667+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.511952+0000 front 2024-04-25T13:24:06.512005+0000 (oldest deadline 2024-04-25T13:24:31.812001+0000) 2024-04-25T13:24:39.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:38.901+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:39.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:38.901+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:39.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:38.690+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:39.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:38.690+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:39.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:38.690+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:39.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:38 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:38.961+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:39.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:38 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:38.961+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:39.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:39 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:39.083+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:39.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:39 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:39.083+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:39.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:39 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:39.083+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:39.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:39 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:39.149+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:39.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:39 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:39.149+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:39.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:39 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:39.340+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:39.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:39 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:39.340+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:39.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:39 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:39.635+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:39.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:39 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:39.635+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.511952+0000 front 2024-04-25T13:24:06.512005+0000 (oldest deadline 2024-04-25T13:24:31.812001+0000) 2024-04-25T13:24:40.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:39 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:39.914+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:40.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:39 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:39.914+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:40.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:39 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:39.708+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:40.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:39 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:39.708+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:40.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:39 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:39.708+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:40.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:39 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:39.999+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:40.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:39 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:39.999+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:40.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:40 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:40.050+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:40.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:40 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:40.050+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:40.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:40 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:40.050+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:40.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:40 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:40.190+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:40.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:40 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:40.190+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:40.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:40 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:40.343+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:40.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:40 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:40.343+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:40.981 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:40 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:40.639+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.811635+0000 front 2024-04-25T13:24:01.811383+0000 (oldest deadline 2024-04-25T13:24:26.511430+0000) 2024-04-25T13:24:40.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:40 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:40.639+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.511952+0000 front 2024-04-25T13:24:06.512005+0000 (oldest deadline 2024-04-25T13:24:31.812001+0000) 2024-04-25T13:24:40.982 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:40 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5[44452]: 2024-04-25T13:24:40.639+0000 7f79e536a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.413879+0000 front 2024-04-25T13:24:16.413699+0000 (oldest deadline 2024-04-25T13:24:39.814411+0000) 2024-04-25T13:24:41.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:40 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:40.946+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:41.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:40 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:40.946+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:41.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:40 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:40.735+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:41.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:40 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:40.735+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:41.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:40 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:40.735+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:41.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:41 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:41.048+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:41.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:41 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:41.048+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:41.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:41 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:41.007+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:41.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:41 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:41.007+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:41.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:41 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:41.007+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:41.644 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:41 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:41.237+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:41.644 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:41 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:41.237+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:41.645 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:41 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:41.383+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:41.645 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:41 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:41.383+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:41.913 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:41 smithi193 podman[69039]: 2024-04-25 13:24:41.644491284 +0000 UTC m=+5.583767149 container died 8b33258ef408a16bf479afbd0843fbb924f174f170f2c59c208f37833a00b482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T13:24:42.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:41 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:41.957+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:42.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:41 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:41.957+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:42.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:41 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:41.714+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:42.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:41 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:41.714+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:42.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:41 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:41.714+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:42.171 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:41 smithi193 podman[69039]: 2024-04-25 13:24:41.987126468 +0000 UTC m=+5.926402327 container cleanup 8b33258ef408a16bf479afbd0843fbb924f174f170f2c59c208f37833a00b482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T13:24:42.171 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:41 smithi193 bash[69039]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5 2024-04-25T13:24:42.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:42 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:42.081+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:42.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:42 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:42.081+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:42.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:41 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:41.983+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:42.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:41 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:41.983+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:42.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:41 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:41.983+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:42.481 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:42 smithi193 podman[69056]: 2024-04-25 13:24:42.171929802 +0000 UTC m=+0.528552711 container remove 8b33258ef408a16bf479afbd0843fbb924f174f170f2c59c208f37833a00b482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:24:42.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:42 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:42.242+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:42.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:42 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:42.242+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:42.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:42 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:42.362+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:42.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:42 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:42.362+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:42.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:42 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:42.362+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:42.908 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:42 smithi193 podman[69152]: 2024-04-25 13:24:42.487592758 +0000 UTC m=+0.018102170 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:42.909 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:42 smithi193 podman[69152]: 2024-04-25 13:24:42.614443913 +0000 UTC m=+0.144953315 container create b8b4ca00b1f7f1e1807f7e50a2b589a2695b4339898c0ecac8a36b0d42721081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T13:24:42.909 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:42 smithi193 podman[69152]: 2024-04-25 13:24:42.8250763 +0000 UTC m=+0.355585693 container init b8b4ca00b1f7f1e1807f7e50a2b589a2695b4339898c0ecac8a36b0d42721081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T13:24:42.909 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:42 smithi193 podman[69152]: 2024-04-25 13:24:42.828185597 +0000 UTC m=+0.358694997 container start b8b4ca00b1f7f1e1807f7e50a2b589a2695b4339898c0ecac8a36b0d42721081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T13:24:43.024 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:42 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:42.981+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:43.025 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:42 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:42.981+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:43.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:42 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:42.748+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:43.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:42 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:42.748+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:43.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:42 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:42.748+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:43.209 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:42 smithi193 podman[69152]: 2024-04-25 13:24:42.908348903 +0000 UTC m=+0.438858297 container attach b8b4ca00b1f7f1e1807f7e50a2b589a2695b4339898c0ecac8a36b0d42721081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T13:24:43.209 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 13:24:43 smithi193 podman[69152]: 2024-04-25 13:24:43.125370848 +0000 UTC m=+0.655880247 container died b8b4ca00b1f7f1e1807f7e50a2b589a2695b4339898c0ecac8a36b0d42721081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:24:43.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:43 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:43.063+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:43.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:43 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:43.063+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:43.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:42 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:42.959+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:43.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:42 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:42.959+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:43.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:42 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:42.959+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:43.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:43 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:43.208+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:43.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:43 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:43.208+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:43.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:43 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:43.326+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:43.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:43 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:43.326+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:43.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:43 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:43.326+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:43.744 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T13:24:43.782 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:24:43.782 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T13:24:43.783 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T13:24:43.783 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.6 2024-04-25T13:24:44.017 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:43 smithi072 systemd[1]: Stopping Ceph osd.6 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:24:44.018 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:43 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:43.744+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:44.018 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:43 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:43.744+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:44.018 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:43 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:43.744+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:44.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:44.014+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:44.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:44.014+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:44.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:44.014+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.488899+0000 front 2024-04-25T13:24:17.488967+0000 (oldest deadline 2024-04-25T13:24:43.388788+0000) 2024-04-25T13:24:44.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:44.107+0000 7f5c979dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:24:44.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:44.107+0000 7f5c979dd700 -1 osd.6 102 *** Got signal Terminated *** 2024-04-25T13:24:44.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:44.107+0000 7f5c979dd700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:24:44.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:44 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:44.085+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:44.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:44 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:44.085+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:44.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:43 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:43.940+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:44.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:43 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:43.940+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:44.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:43 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:43.940+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:44.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:44 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:44.163+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:44.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:44 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:44.163+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:44.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:44 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:44.323+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:44.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:44 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:44.323+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:44.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:44 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:44.323+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:45.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:44.990+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:45.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:44.990+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:45.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:44.990+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.488899+0000 front 2024-04-25T13:24:17.488967+0000 (oldest deadline 2024-04-25T13:24:43.388788+0000) 2024-04-25T13:24:45.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:44.792+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:45.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:44.792+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:45.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:44 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:44.792+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:45.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:45 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:45.042+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:45.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:45 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:45.042+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:45.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:44 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:44.986+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:45.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:44 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:44.986+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:45.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:44 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:44.986+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:45.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:45 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:45.123+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:45.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:45 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:45.123+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:45.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:45 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:45.303+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:45.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:45 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:45.303+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:45.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:45 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:45.303+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:46.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:45 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:45.771+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:46.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:45 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:45.771+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:46.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:45 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:45.771+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:46.026 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:45 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:45.998+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:46.026 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:45 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:45.998+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:46.027 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:45 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:45.998+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.488899+0000 front 2024-04-25T13:24:17.488967+0000 (oldest deadline 2024-04-25T13:24:43.388788+0000) 2024-04-25T13:24:46.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:45 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:45.993+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:46.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:45 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:45.993+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:46.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:45 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:45.974+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:46.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:45 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:45.974+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:46.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:45 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:45.974+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:46.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:46.169+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:46.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:46.169+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:46.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:46.329+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:46.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:46.329+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:46.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:46 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:46.329+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:47.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:46 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:46.963+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:47.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:46 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:46.963+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:47.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:46 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:46.963+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.488899+0000 front 2024-04-25T13:24:17.488967+0000 (oldest deadline 2024-04-25T13:24:43.388788+0000) 2024-04-25T13:24:47.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:46 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:46.804+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:47.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:46 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:46.804+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:47.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:46 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:46.804+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:47.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:46 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:46.804+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:47.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:46.967+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:47.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:46 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:46.967+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:47.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:47 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:47.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:47.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:47 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:47.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:47.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:47 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:47.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:47.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:47 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:47.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:47.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:47 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:47.138+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:47.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:47 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:47.138+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:47.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:47 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:47.292+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:47.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:47 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:47.292+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:47.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:47 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:47.292+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:47.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:47 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:47.292+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:48.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:47 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:47.923+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:48.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:47 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:47.923+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:48.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:47 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:47.923+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.488899+0000 front 2024-04-25T13:24:17.488967+0000 (oldest deadline 2024-04-25T13:24:43.388788+0000) 2024-04-25T13:24:48.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:47 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:47.836+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:48.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:47 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:47.836+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:48.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:47 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:47.836+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:48.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:47 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:47.836+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:48.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:47 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:47.956+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:48.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:47 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:47.956+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:48.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:47 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:47.956+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:48.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:48 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:48.026+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:48.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:48 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:48.026+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:48.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:48 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:48.026+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:48.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:48 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:48.026+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:48.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:48 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:48.131+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:48.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:48 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:48.131+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:48.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:48 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:48.254+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:48.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:48 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:48.254+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:48.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:48 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:48.254+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:48.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:48 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:48.254+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:49.274 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:48 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:48.788+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:49.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:48 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:48.788+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:49.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:48 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:48.788+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:49.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:48 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:48.788+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:49.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:48 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:48.903+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.386086+0000 front 2024-04-25T13:23:59.386134+0000 (oldest deadline 2024-04-25T13:24:23.486145+0000) 2024-04-25T13:24:49.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:48 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:48.903+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.287755+0000 front 2024-04-25T13:24:09.287705+0000 (oldest deadline 2024-04-25T13:24:31.587762+0000) 2024-04-25T13:24:49.275 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:48 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:48.903+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.488899+0000 front 2024-04-25T13:24:17.488967+0000 (oldest deadline 2024-04-25T13:24:43.388788+0000) 2024-04-25T13:24:49.276 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:48 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6[57763]: 2024-04-25T13:24:48.903+0000 7f5c8ffc9700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:23.389337+0000 front 2024-04-25T13:24:23.389268+0000 (oldest deadline 2024-04-25T13:24:48.689478+0000) 2024-04-25T13:24:49.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:48 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:48.922+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:49.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:48 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:48.922+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:49.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:48 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:48.922+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:49.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:49.001+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:49.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:49.001+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:49.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:49.001+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:49.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:49.001+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:49.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:49 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:49.103+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:49.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:49 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:49.103+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:49.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:49 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:49.211+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:49.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:49 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:49.211+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:49.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:49 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:49.211+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:49.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:49 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:49.211+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:49.827 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:49 smithi072 podman[87779]: 2024-04-25 13:24:49.561731671 +0000 UTC m=+5.575679315 container died eae38ecbc546dea7ffbe2e35143ed6bb4655d06c746f52ba887a22a332a227e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T13:24:49.828 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:49 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:49.788+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:49.828 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:49 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:49.788+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:49.828 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:49 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:49.788+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:49.828 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:49 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:49.788+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:50.195 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:49 smithi072 podman[87779]: 2024-04-25 13:24:49.927024913 +0000 UTC m=+5.940972546 container cleanup eae38ecbc546dea7ffbe2e35143ed6bb4655d06c746f52ba887a22a332a227e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:24:50.196 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:49 smithi072 bash[87779]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6 2024-04-25T13:24:50.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:49.971+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:50.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:49.971+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:50.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:49.971+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:50.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:49.996+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:50.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:49.996+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:50.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:49.996+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:50.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:49 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:49.996+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:50.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:50 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:50.146+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:50.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:50 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:50.146+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:50.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:50 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:50.187+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:50.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:50 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:50.187+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:50.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:50 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:50.187+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:50.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:50 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:50.187+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:50.524 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:50 smithi072 podman[87797]: 2024-04-25 13:24:50.196000724 +0000 UTC m=+0.631490462 container remove eae38ecbc546dea7ffbe2e35143ed6bb4655d06c746f52ba887a22a332a227e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T13:24:50.861 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:50 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:50.745+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:50.862 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:50 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:50.745+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:50.862 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:50 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:50.745+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:50.862 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:50 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:50.745+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:50.862 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:50 smithi072 podman[87889]: 2024-04-25 13:24:50.439320603 +0000 UTC m=+0.017906947 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:50.862 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:50 smithi072 podman[87889]: 2024-04-25 13:24:50.55762288 +0000 UTC m=+0.136209223 container create 89eb67efed6470c423fa9d32dc834a7e8f5ceb53844cd510f2c23228463c87e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T13:24:50.862 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:50 smithi072 podman[87889]: 2024-04-25 13:24:50.778265859 +0000 UTC m=+0.356852205 container init 89eb67efed6470c423fa9d32dc834a7e8f5ceb53844cd510f2c23228463c87e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:24:50.863 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:50 smithi072 podman[87889]: 2024-04-25 13:24:50.781321618 +0000 UTC m=+0.359907962 container start 89eb67efed6470c423fa9d32dc834a7e8f5ceb53844cd510f2c23228463c87e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:24:50.863 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:50 smithi072 podman[87889]: 2024-04-25 13:24:50.861958971 +0000 UTC m=+0.440545323 container attach 89eb67efed6470c423fa9d32dc834a7e8f5ceb53844cd510f2c23228463c87e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T13:24:51.274 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 25 13:24:51 smithi072 podman[87889]: 2024-04-25 13:24:51.07135149 +0000 UTC m=+0.649937832 container died 89eb67efed6470c423fa9d32dc834a7e8f5ceb53844cd510f2c23228463c87e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T13:24:51.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:50 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:50.988+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:51.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:50 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:50.988+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:51.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:50 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:50.988+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:51.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:51.004+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:51.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:51.004+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:51.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:51.004+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:51.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:51.004+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:51.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:51 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:51.139+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:51.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:51 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:51.139+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:51.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:51 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:51.139+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:24:51.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:51 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:51.197+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:51.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:51 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:51.197+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:51.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:51 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:51.197+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:51.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:51 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:51.197+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:51.652 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T13:24:51.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:24:51.691 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T13:24:51.691 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T13:24:51.691 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.7 2024-04-25T13:24:51.995 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:51 smithi164 systemd[1]: Stopping Ceph osd.7 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:24:52.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:51 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:51.794+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:52.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:51 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:51.794+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:52.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:51 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:51.794+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:52.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:51 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:51.794+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:52.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:52.039+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:52.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:52.039+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:52.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:52.039+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:52.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:52.039+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:52.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:51.993+0000 7f76b8296700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:24:52.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:51.993+0000 7f76b8296700 -1 osd.7 102 *** Got signal Terminated *** 2024-04-25T13:24:52.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:51.993+0000 7f76b8296700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:24:52.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:52.001+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:52.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:52.001+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:52.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:52.001+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:52.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:52 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:52.170+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:52.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:52 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:52.170+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:52.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:52 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:52.170+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:24:52.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:52 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:52.221+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:52.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:52 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:52.221+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:52.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:52 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:52.221+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:52.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:52 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:52.221+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:53.274 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:52 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:52.829+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:53.274 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:52 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:52.829+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:53.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:52 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:52.829+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:53.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:52 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:52.829+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:53.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:52.980+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:53.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:52.980+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:53.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:52 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:52.980+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:53.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:53 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:53.082+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:53.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:53 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:53.082+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:53.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:53 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:53.082+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:53.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:53 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:53.082+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:53.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:53 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:53.147+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:53.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:53 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:53.147+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:53.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:53 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:53.147+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:24:53.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:53 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:53.186+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:53.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:53 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:53.186+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:53.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:53 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:53.186+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:53.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:53 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:53.186+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:54.274 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:53 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:53.797+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:54.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:53 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:53.797+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:54.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:53 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:53.797+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:54.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:53 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:53.797+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:54.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:53 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:53.946+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:54.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:53 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:53.946+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:54.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:53 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:53.946+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:54.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:54 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:54.044+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:54.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:54 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:54.044+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:54.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:54 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:54.044+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:54.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:54 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:54.044+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:54.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:54 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:54.183+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:54.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:54 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:54.183+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:54.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:54 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:54.183+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:24:54.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:54 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:54.162+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:54.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:54 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:54.162+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:54.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:54 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:54.162+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:54.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:54 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:54.162+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:55.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:54 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:54.763+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:55.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:54 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:54.763+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:55.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:54 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:54.763+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:55.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:54 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:54.763+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:55.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:54 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:54.914+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:55.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:54 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:54.915+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:55.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:54 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:54.915+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:55.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:55.010+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:55.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:55.010+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:55.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:55.010+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:55.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:55.010+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:55.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:55 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:55.210+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:55.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:55 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:55.210+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:55.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:55 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:55.210+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:55.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:55 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:55.210+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:55.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:55 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:55.210+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:55.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:55 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:55.210+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:55.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:55 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:55.210+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:24:55.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:55 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:55.210+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:24:56.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:55 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:55.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:56.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:55 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:55.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:56.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:55 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:55.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:56.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:55 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:55.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:56.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:55.934+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:56.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:55.934+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:56.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:55.934+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:56.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:55.934+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.895402+0000 front 2024-04-25T13:24:30.895445+0000 (oldest deadline 2024-04-25T13:24:54.995550+0000) 2024-04-25T13:24:56.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:56.051+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:56.309 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:56.051+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:56.309 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:56.051+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:56.309 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:56.051+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:56.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:56.242+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:56.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:56.242+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:56.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:56.242+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:24:56.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:56.242+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:24:56.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:56.161+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:56.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:56.161+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:56.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:56.161+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:56.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:56 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:56.161+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:57.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:56 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:56.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:57.109 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:56 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:56.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:57.110 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:56 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:56.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:57.110 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:56 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:56.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:57.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:56.895+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.793119+0000 front 2024-04-25T13:24:09.793095+0000 (oldest deadline 2024-04-25T13:24:35.093261+0000) 2024-04-25T13:24:57.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:56.895+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.093849+0000 front 2024-04-25T13:24:15.093710+0000 (oldest deadline 2024-04-25T13:24:38.593735+0000) 2024-04-25T13:24:57.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:56.895+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:22.094723+0000 front 2024-04-25T13:24:22.094646+0000 (oldest deadline 2024-04-25T13:24:47.394543+0000) 2024-04-25T13:24:57.308 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:56 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7[50249]: 2024-04-25T13:24:56.895+0000 7f76b488a700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.895402+0000 front 2024-04-25T13:24:30.895445+0000 (oldest deadline 2024-04-25T13:24:54.995550+0000) 2024-04-25T13:24:57.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.026+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:57.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.026+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:57.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.026+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:57.309 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.026+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:57.309 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.026+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:24:57.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:57.122+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:57.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:57.122+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:57.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:57.122+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:57.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:57.122+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:57.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:57.217+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:57.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:57.217+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:57.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:57.217+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:24:57.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:57 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:57.217+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:24:57.637 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:57 smithi164 podman[68316]: 2024-04-25 13:24:57.385428845 +0000 UTC m=+5.500624135 container died aa6d0b0f21287142c463f882b925c0480d694922f2e5dda8c366bc199ae3acd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T13:24:57.903 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:57 smithi164 podman[68316]: 2024-04-25 13:24:57.719752093 +0000 UTC m=+5.834947428 container cleanup aa6d0b0f21287142c463f882b925c0480d694922f2e5dda8c366bc199ae3acd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:24:57.904 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:57 smithi164 bash[68316]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7 2024-04-25T13:24:58.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:57.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:58.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:57.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:58.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:57.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:58.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:57.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:58.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:57 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:57.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:24:58.307 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:57 smithi164 podman[68334]: 2024-04-25 13:24:57.904016893 +0000 UTC m=+0.512693488 container remove aa6d0b0f21287142c463f882b925c0480d694922f2e5dda8c366bc199ae3acd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:24:58.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.987+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:58.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.987+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:58.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.987+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:58.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.987+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:58.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:57 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:57.987+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:24:58.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:58.097+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:58.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:58.097+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:58.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:58.097+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:58.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:58.097+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:58.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:58.166+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:58.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:58.166+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:58.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:58.166+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:24:58.483 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:58 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:58.167+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:24:58.567 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:58 smithi164 podman[68425]: 2024-04-25 13:24:58.313057735 +0000 UTC m=+0.093818926 container create a62d2e7ed4f0ba43ec20c8e2c8daa84c06ef6c95eab250f14f0b413277b81fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T13:24:58.567 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:58 smithi164 podman[68425]: 2024-04-25 13:24:58.237342487 +0000 UTC m=+0.018103654 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:58.567 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:58 smithi164 podman[68425]: 2024-04-25 13:24:58.567650861 +0000 UTC m=+0.348412024 container init a62d2e7ed4f0ba43ec20c8e2c8daa84c06ef6c95eab250f14f0b413277b81fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T13:24:58.882 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:58 smithi164 podman[68425]: 2024-04-25 13:24:58.57087209 +0000 UTC m=+0.351633237 container start a62d2e7ed4f0ba43ec20c8e2c8daa84c06ef6c95eab250f14f0b413277b81fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:24:58.882 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:58 smithi164 podman[68425]: 2024-04-25 13:24:58.690579181 +0000 UTC m=+0.471340340 container attach a62d2e7ed4f0ba43ec20c8e2c8daa84c06ef6c95eab250f14f0b413277b81fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T13:24:59.168 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:58 smithi164 podman[68425]: 2024-04-25 13:24:58.883153779 +0000 UTC m=+0.663914943 container died a62d2e7ed4f0ba43ec20c8e2c8daa84c06ef6c95eab250f14f0b413277b81fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T13:24:59.168 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.003+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:24:59.168 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.003+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:24:59.169 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.003+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:24:59.169 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.003+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:24:59.169 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.003+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:24:59.274 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:58 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:58.784+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:24:59.274 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:58 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:58.784+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:24:59.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:58 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:58.784+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:24:59.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:58 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:58.784+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:24:59.275 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:58 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:58.784+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:24:59.423 INFO:journalctl@ceph.osd.7.smithi164.stdout:Apr 25 13:24:59 smithi164 podman[68451]: 2024-04-25 13:24:59.410601837 +0000 UTC m=+0.521662626 container remove a62d2e7ed4f0ba43ec20c8e2c8daa84c06ef6c95eab250f14f0b413277b81fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-25T13:24:59.428 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T13:24:59.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:24:59.465 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T13:24:59.465 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T13:24:59.466 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.8 2024-04-25T13:24:59.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:59.182+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:24:59.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:59.182+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:24:59.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:59.182+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:24:59.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:59.182+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:24:59.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:59.118+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:24:59.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:59.118+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:24:59.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:59.118+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:24:59.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:59.118+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:24:59.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:24:59.118+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:24:59.755 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:59 smithi193 systemd[1]: Stopping Ceph osd.8 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:00.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:59 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:59.745+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:00.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:59 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:59.745+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:00.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:59 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:59.745+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:00.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:59 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:59.745+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:00.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:24:59 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:24:59.745+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:00.075 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:59.755+0000 7fde34116700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:00.075 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:59.755+0000 7fde34116700 -1 osd.8 102 *** Got signal Terminated *** 2024-04-25T13:25:00.075 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:24:59 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:24:59.755+0000 7fde34116700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:25:00.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.960+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:00.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.960+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:00.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.960+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:00.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.960+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:00.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:24:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:24:59.960+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:00.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:00.159+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:25:00.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:00.159+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:25:00.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:00.159+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:25:00.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:00.159+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:25:00.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:00.074+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:00.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:00.074+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:00.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:00.074+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:00.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:00.074+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:00.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:00 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:00.074+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:01.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:00.737+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:01.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:00.737+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:01.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:00.737+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:01.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:00.737+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:01.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:00 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:00.737+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:01.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:00 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:00.978+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:01.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:00 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:00.978+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:01.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:00 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:00.978+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:01.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:00 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:00.978+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:01.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:00 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:00.978+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:01.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:01.115+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:25:01.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:01.115+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:25:01.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:01.115+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:25:01.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:01.115+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:25:01.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:01.065+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:01.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:01.065+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:01.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:01.065+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:01.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:01.065+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:01.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:01 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:01.065+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:02.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:01.699+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:02.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:01.699+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:02.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:01.699+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:02.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:01.699+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:02.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:01 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:01.699+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:02.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.007+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:02.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.007+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:02.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.007+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:02.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.007+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:02.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.007+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:02.481 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:02.154+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:25:02.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:02.154+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:25:02.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:02.154+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:25:02.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:02.154+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:25:02.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:02.059+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:02.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:02.059+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:02.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:02.059+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:02.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:02.059+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:02.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:02 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:02.059+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:02.926 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:02.661+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:02.926 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:02.661+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:02.926 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:02.661+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:02.926 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:02.661+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:02.926 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:02.661+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:02.926 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:02 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:02.661+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:03.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.983+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:03.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.983+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:03.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.983+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:03.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.983+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:03.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:02 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:02.983+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:03.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:03.196+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:25:03.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:03.196+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:25:03.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:03.196+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:25:03.482 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:03.196+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:25:03.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:03.033+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:03.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:03.033+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:03.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:03.033+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:03.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:03.033+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:03.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:03 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:03.033+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:04.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:03.658+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:04.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:03.658+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:04.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:03.658+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:04.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:03.658+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:04.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:03.658+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:04.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:03 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:03.658+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:04.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:04.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:04.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:04.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:04.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:04.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.005+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:04.383 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:04.236+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:58.141162+0000 front 2024-04-25T13:23:58.141038+0000 (oldest deadline 2024-04-25T13:24:23.941656+0000) 2024-04-25T13:25:04.383 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:04.236+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.842772+0000 front 2024-04-25T13:24:09.843071+0000 (oldest deadline 2024-04-25T13:24:32.043360+0000) 2024-04-25T13:25:04.383 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:04.236+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.945855+0000 front 2024-04-25T13:24:24.945876+0000 (oldest deadline 2024-04-25T13:24:50.245614+0000) 2024-04-25T13:25:04.383 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8[51341]: 2024-04-25T13:25:04.236+0000 7fde2c702700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.746530+0000 front 2024-04-25T13:24:30.746550+0000 (oldest deadline 2024-04-25T13:24:54.246620+0000) 2024-04-25T13:25:04.384 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:04.001+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:04.384 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:04.001+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:04.384 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:04.001+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:04.384 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:04.001+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:04.384 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:04.001+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:05.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:04.697+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:05.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:04.697+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:05.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:04.697+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:05.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:04.697+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:05.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:04.697+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:05.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:04 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:04.697+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:05.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.990+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:05.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.990+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:05.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.990+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:05.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.990+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:05.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.990+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:05.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:04 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:04.990+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:05.394 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:05 smithi193 podman[69258]: 2024-04-25 13:25:05.264597302 +0000 UTC m=+5.604740308 container died ef88b21be693e0e5303e74df3a2f92fab755dd9d6b303a327c10ed142afffc3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T13:25:05.394 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:05 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:05.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:05.394 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:05 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:05.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:05.394 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:05 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:05.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:05.395 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:05 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:05.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:05.395 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:05 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:05.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:05.395 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:05 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:05.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:05.982 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:05 smithi193 podman[69258]: 2024-04-25 13:25:05.663552064 +0000 UTC m=+6.003695067 container cleanup ef88b21be693e0e5303e74df3a2f92fab755dd9d6b303a327c10ed142afffc3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T13:25:05.982 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:05 smithi193 bash[69258]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8 2024-04-25T13:25:05.982 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:05 smithi193 podman[69276]: 2024-04-25 13:25:05.88138991 +0000 UTC m=+0.615096224 container remove ef88b21be693e0e5303e74df3a2f92fab755dd9d6b303a327c10ed142afffc3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T13:25:06.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:05.703+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:06.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:05.703+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:06.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:05.703+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:06.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:05.703+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:06.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:05.703+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:06.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:05 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:05.703+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:06.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:05 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:05.943+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:06.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:05 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:05.943+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:06.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:05 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:05.943+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:06.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:05 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:05.943+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:06.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:05 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:05.943+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:06.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:05 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:05.943+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:06.362 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:06 smithi193 podman[69374]: 2024-04-25 13:25:06.132815611 +0000 UTC m=+0.018349994 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:25:06.362 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:06 smithi193 podman[69374]: 2024-04-25 13:25:06.248830293 +0000 UTC m=+0.134364657 container create 26ea7abc736ac35543eb87b77aceaf12874af4494f97b93940872c77f3168844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:25:06.362 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.012+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:06.362 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.012+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:06.362 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.012+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:06.362 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.012+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:06.363 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.012+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:06.363 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.012+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:06.631 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:06 smithi193 podman[69374]: 2024-04-25 13:25:06.467986723 +0000 UTC m=+0.353521111 container init 26ea7abc736ac35543eb87b77aceaf12874af4494f97b93940872c77f3168844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:25:06.631 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:06 smithi193 podman[69374]: 2024-04-25 13:25:06.47153303 +0000 UTC m=+0.357067403 container start 26ea7abc736ac35543eb87b77aceaf12874af4494f97b93940872c77f3168844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-04-25T13:25:06.631 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:06 smithi193 podman[69374]: 2024-04-25 13:25:06.534449915 +0000 UTC m=+0.419984294 container attach 26ea7abc736ac35543eb87b77aceaf12874af4494f97b93940872c77f3168844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T13:25:06.981 INFO:journalctl@ceph.osd.8.smithi193.stdout:Apr 25 13:25:06 smithi193 podman[69374]: 2024-04-25 13:25:06.771784174 +0000 UTC m=+0.657318542 container died 26ea7abc736ac35543eb87b77aceaf12874af4494f97b93940872c77f3168844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T13:25:07.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:06.731+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:07.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:06.731+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:07.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:06.731+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:07.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:06.731+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:07.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:06.731+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:07.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:06 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:06.731+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:07.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:06 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:06.900+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:07.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:06 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:06.900+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:07.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:06 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:06.900+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:07.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:06 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:06.900+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:07.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:06 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:06.900+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:07.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:06 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:06.900+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:07.338 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.983+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:07.338 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.983+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:07.338 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.983+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:07.338 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.983+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:07.338 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.983+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:07.338 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:06 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:06.983+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:07.366 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T13:25:07.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:07.447 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T13:25:07.447 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T13:25:07.448 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.9 2024-04-25T13:25:07.732 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 systemd[1]: Stopping Ceph osd.9 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:08.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:07.731+0000 7f34308dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:08.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:07.731+0000 7f34308dd700 -1 osd.9 102 *** Got signal Terminated *** 2024-04-25T13:25:08.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:07.731+0000 7f34308dd700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:25:08.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:07.747+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:08.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:07.747+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:08.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:07.747+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:08.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:07.747+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:08.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:07.747+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:08.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:07 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:07.747+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:08.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:07 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:07.940+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:08.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:07 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:07.940+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:08.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:07 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:07.940+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:08.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:07 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:07.940+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:08.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:07 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:07.940+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:08.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:07 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:07.940+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:08.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:07 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:07.914+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:08.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:07 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:07.914+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:08.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:07 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:07.914+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:08.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:07 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:07.914+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:08.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:07 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:07.914+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:08.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:07 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:07.914+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:09.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:08 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:08.712+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:09.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:08 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:08.712+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:09.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:08 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:08.712+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:09.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:08 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:08.712+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:09.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:08 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:08.712+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:09.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:08 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:08.712+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:09.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:08 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:08.887+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:09.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:08 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:08.887+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:09.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:08 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:08.887+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:09.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:08 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:08.887+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:09.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:08 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:08.887+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:09.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:08 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:08.887+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:09.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:08 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:08.988+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:09.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:08 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:08.988+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:09.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:08 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:08.988+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:09.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:08 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:08.988+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:09.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:08 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:08.988+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:09.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:08 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:08.988+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:10.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:09.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:10.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:09.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:10.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:09.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:10.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:09.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:10.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:09.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:10.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:09 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:09.757+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:10.231 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:09 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:09.951+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:10.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:09 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:09.951+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:10.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:09 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:09.951+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:10.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:09 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:09.951+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:10.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:09 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:09.951+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:10.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:09 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:09.951+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:10.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:09 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:09.897+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:10.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:09 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:09.897+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:10.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:09 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:09.897+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:10.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:09 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:09.897+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:10.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:09 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:09.897+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:10.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:09 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:09.897+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:11.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:10 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:10.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:11.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:10 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:10.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:11.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:10 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:10.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:11.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:10 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:10.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:11.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:10 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:10.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:11.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:10 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:10.710+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:11.057 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:10 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:10.866+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:11.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:10 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:10.866+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:11.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:10 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:10.866+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:11.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:10 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:10.866+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:11.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:10 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:10.866+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:11.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:10 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:10.866+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:11.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:10 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:10.990+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:11.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:10 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:10.990+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:11.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:10 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:10.990+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:11.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:10 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:10.990+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:11.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:10 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:10.990+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:11.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:10 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:10.990+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:12.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:11 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:11.674+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:12.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:11 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:11.674+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:12.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:11 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:11.674+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:12.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:11 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:11.674+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:12.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:11 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:11.674+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:12.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:11 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:11.674+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:12.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:11 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:11.945+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:12.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:11 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:11.945+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:12.233 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:11 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:11.945+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:12.233 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:11 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:11.945+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:12.233 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:11 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:11.945+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:12.233 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:11 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:11.945+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:12.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:11 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:11.829+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:12.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:11 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:11.829+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:12.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:11 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:11.829+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:12.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:11 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:11.829+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:12.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:11 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:11.829+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:12.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:11 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:11.829+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:12.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:11 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:11.829+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-25T13:24:45.317107+0000 front 2024-04-25T13:24:45.317240+0000 (oldest deadline 2024-04-25T13:25:11.217031+0000) 2024-04-25T13:25:13.024 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:12 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:12.679+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:23:59.805659+0000 front 2024-04-25T13:23:59.805612+0000 (oldest deadline 2024-04-25T13:24:24.505680+0000) 2024-04-25T13:25:13.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:12 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:12.679+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:09.007820+0000 front 2024-04-25T13:24:09.007803+0000 (oldest deadline 2024-04-25T13:24:33.707826+0000) 2024-04-25T13:25:13.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:12 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:12.679+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:16.008889+0000 front 2024-04-25T13:24:16.008956+0000 (oldest deadline 2024-04-25T13:24:38.308812+0000) 2024-04-25T13:25:13.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:12 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:12.679+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.710476+0000 front 2024-04-25T13:24:24.710442+0000 (oldest deadline 2024-04-25T13:24:46.410524+0000) 2024-04-25T13:25:13.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:12 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:12.679+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:33.412297+0000 front 2024-04-25T13:24:33.412323+0000 (oldest deadline 2024-04-25T13:24:57.512206+0000) 2024-04-25T13:25:13.025 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:12 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9[64806]: 2024-04-25T13:25:12.679+0000 7f342ced1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.413295+0000 front 2024-04-25T13:24:40.413238+0000 (oldest deadline 2024-04-25T13:25:02.613734+0000) 2024-04-25T13:25:13.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:12 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:12.926+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:13.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:12 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:12.926+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:13.233 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:12 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:12.926+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:13.233 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:12 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:12.926+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:13.233 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:12 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:12.926+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:13.233 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:12 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:12.926+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:13.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:12 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:12.844+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:13.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:12 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:12.844+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:13.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:12 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:12.844+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:13.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:12 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:12.844+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:13.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:12 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:12.844+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:13.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:12 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:12.844+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:13.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:12 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:12.844+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-25T13:24:45.317107+0000 front 2024-04-25T13:24:45.317240+0000 (oldest deadline 2024-04-25T13:25:11.217031+0000) 2024-04-25T13:25:13.461 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:13 smithi072 podman[88010]: 2024-04-25 13:25:13.199301255 +0000 UTC m=+5.559344900 container died 9754ac952a63d78ca5ee101c70f1e3eac447b719c0252abe372d0c9c633c2994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T13:25:13.774 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:13 smithi072 podman[88010]: 2024-04-25 13:25:13.560985855 +0000 UTC m=+5.921029490 container cleanup 9754ac952a63d78ca5ee101c70f1e3eac447b719c0252abe372d0c9c633c2994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T13:25:13.775 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:13 smithi072 bash[88010]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9 2024-04-25T13:25:14.181 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:13 smithi072 podman[88028]: 2024-04-25 13:25:13.871765786 +0000 UTC m=+0.668032544 container remove 9754ac952a63d78ca5ee101c70f1e3eac447b719c0252abe372d0c9c633c2994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T13:25:14.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:13 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:13.927+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:14.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:13 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:13.927+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:14.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:13 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:13.927+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:14.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:13 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:13.927+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:14.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:13 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:13.927+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:14.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:13 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:13.927+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:14.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:13 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:13.814+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:14.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:13 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:13.814+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:14.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:13 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:13.814+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:14.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:13 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:13.814+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:14.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:13 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:13.814+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:14.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:13 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:13.814+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:14.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:13 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:13.814+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-25T13:24:45.317107+0000 front 2024-04-25T13:24:45.317240+0000 (oldest deadline 2024-04-25T13:25:11.217031+0000) 2024-04-25T13:25:14.512 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:14 smithi072 podman[88115]: 2024-04-25 13:25:14.088254196 +0000 UTC m=+0.017961276 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:25:14.512 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:14 smithi072 podman[88115]: 2024-04-25 13:25:14.191620451 +0000 UTC m=+0.121327518 container create 6c5556611b039065337205942659d087acd7d1bbd2cd19ad740634db74eb2ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T13:25:14.512 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:14 smithi072 podman[88115]: 2024-04-25 13:25:14.420464084 +0000 UTC m=+0.350171150 container init 6c5556611b039065337205942659d087acd7d1bbd2cd19ad740634db74eb2ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-25T13:25:14.513 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:14 smithi072 podman[88115]: 2024-04-25 13:25:14.424409676 +0000 UTC m=+0.354116743 container start 6c5556611b039065337205942659d087acd7d1bbd2cd19ad740634db74eb2ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T13:25:14.513 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:14 smithi072 podman[88115]: 2024-04-25 13:25:14.512445115 +0000 UTC m=+0.442152199 container attach 6c5556611b039065337205942659d087acd7d1bbd2cd19ad740634db74eb2ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:25:14.774 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 25 13:25:14 smithi072 podman[88115]: 2024-04-25 13:25:14.712713682 +0000 UTC m=+0.642420750 container died 6c5556611b039065337205942659d087acd7d1bbd2cd19ad740634db74eb2ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T13:25:15.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:14 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:14.912+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:15.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:14 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:14.912+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:15.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:14 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:14.912+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:15.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:14 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:14.912+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:15.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:14 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:14.912+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:15.233 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:14 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:14.912+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:15.292 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T13:25:15.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:14 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:14.820+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:15.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:14 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:14.820+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:15.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:14 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:14.820+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:15.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:14 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:14.820+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:15.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:14 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:14.820+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:15.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:14 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:14.820+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:15.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:14 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:14.820+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-25T13:24:45.317107+0000 front 2024-04-25T13:24:45.317240+0000 (oldest deadline 2024-04-25T13:25:11.217031+0000) 2024-04-25T13:25:15.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:15.333 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T13:25:15.333 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T13:25:15.333 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.10 2024-04-25T13:25:15.807 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 systemd[1]: Stopping Ceph osd.10 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:15.807 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.603+0000 7fef949c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:15.807 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.603+0000 7fef949c5700 -1 osd.10 102 *** Got signal Terminated *** 2024-04-25T13:25:15.807 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.603+0000 7fef949c5700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:25:16.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:15 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:15.944+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:16.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:15 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:15.944+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:16.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:15 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:15.944+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:16.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:15 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:15.944+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:16.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:15 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:15.944+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:16.232 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:15 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:15.944+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:16.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.846+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:16.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.846+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:16.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.846+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:16.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.846+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:16.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.846+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:16.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.846+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:16.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:15 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:15.846+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-25T13:24:45.317107+0000 front 2024-04-25T13:24:45.317240+0000 (oldest deadline 2024-04-25T13:25:11.217031+0000) 2024-04-25T13:25:17.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:16 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:16.840+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:17.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:16 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:16.840+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:17.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:16 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:16.840+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:17.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:16 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:16.840+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:17.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:16 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:16.840+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:17.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:16 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:16.840+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:17.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:16 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:16.840+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-25T13:24:45.317107+0000 front 2024-04-25T13:24:45.317240+0000 (oldest deadline 2024-04-25T13:25:11.217031+0000) 2024-04-25T13:25:17.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:16 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:16.982+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:17.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:16 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:16.982+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:17.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:16 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:16.982+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:17.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:16 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:16.982+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:17.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:16 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:16.982+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:17.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:16 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:16.982+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:18.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:17 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:17.838+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:18.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:17 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:17.838+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:18.307 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:17 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:17.838+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:18.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:17 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:17.838+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:18.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:17 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:17.838+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:18.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:17 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:17.838+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:18.308 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:17 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:17.838+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-25T13:24:45.317107+0000 front 2024-04-25T13:24:45.317240+0000 (oldest deadline 2024-04-25T13:25:11.217031+0000) 2024-04-25T13:25:18.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:18.015+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:18.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:18.015+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:18.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:18.015+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:18.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:18.015+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:18.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:18.015+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:18.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:18 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:18.015+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:19.057 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:18 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:18.801+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:19.057 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:18 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:18.801+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:19.057 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:18 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:18.801+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:19.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:18 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:18.801+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:19.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:18 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:18.801+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:19.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:18 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:18.801+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:19.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:18 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:18.801+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-25T13:24:45.317107+0000 front 2024-04-25T13:24:45.317240+0000 (oldest deadline 2024-04-25T13:25:11.217031+0000) 2024-04-25T13:25:19.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:19 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:19.027+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:19.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:19 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:19.027+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:19.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:19 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:19.027+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:19.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:19 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:19.027+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:19.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:19 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:19.027+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:19.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:19 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:19.027+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:19.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:19 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:19.027+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:20.057 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:19 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:19.785+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.809367+0000 front 2024-04-25T13:24:01.809467+0000 (oldest deadline 2024-04-25T13:24:27.709323+0000) 2024-04-25T13:25:20.057 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:19 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:19.785+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:07.709833+0000 front 2024-04-25T13:24:07.709935+0000 (oldest deadline 2024-04-25T13:24:31.209948+0000) 2024-04-25T13:25:20.057 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:19 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:19.785+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:15.211444+0000 front 2024-04-25T13:24:15.211276+0000 (oldest deadline 2024-04-25T13:24:38.711239+0000) 2024-04-25T13:25:20.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:19 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:19.785+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:24.612639+0000 front 2024-04-25T13:24:24.612461+0000 (oldest deadline 2024-04-25T13:24:46.912418+0000) 2024-04-25T13:25:20.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:19 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:19.785+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:32.614427+0000 front 2024-04-25T13:24:32.614442+0000 (oldest deadline 2024-04-25T13:24:56.714493+0000) 2024-04-25T13:25:20.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:19 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:19.785+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:40.116037+0000 front 2024-04-25T13:24:40.116217+0000 (oldest deadline 2024-04-25T13:25:03.616074+0000) 2024-04-25T13:25:20.058 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:19 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10[56868]: 2024-04-25T13:25:19.785+0000 7fef8cfb1700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-25T13:24:45.317107+0000 front 2024-04-25T13:24:45.317240+0000 (oldest deadline 2024-04-25T13:25:11.217031+0000) 2024-04-25T13:25:20.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:20.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:20.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:20.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:20.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:20.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:20.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:20.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:20.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:20.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:20.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:20.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:20.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:20 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:20.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:21.295 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:21 smithi164 podman[68542]: 2024-04-25 13:25:21.011888742 +0000 UTC m=+5.484913470 container died 631450c60eb523a5eb1656f12189f3f4512d14c01ce20c6f5c522cbe84326a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-25T13:25:21.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:21.067+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:21.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:21.067+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:21.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:21.067+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:21.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:21.067+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:21.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:21.067+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:21.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:21.067+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:21.484 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:21 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:21.067+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:21.557 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:21 smithi164 podman[68542]: 2024-04-25 13:25:21.395447299 +0000 UTC m=+5.868472034 container cleanup 631450c60eb523a5eb1656f12189f3f4512d14c01ce20c6f5c522cbe84326a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:25:21.557 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:21 smithi164 bash[68542]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10 2024-04-25T13:25:21.962 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:21 smithi164 podman[68559]: 2024-04-25 13:25:21.646504205 +0000 UTC m=+0.623826312 container remove 631450c60eb523a5eb1656f12189f3f4512d14c01ce20c6f5c522cbe84326a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T13:25:22.225 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:21 smithi164 podman[68648]: 2024-04-25 13:25:21.864928664 +0000 UTC m=+0.018112171 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:25:22.225 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:21 smithi164 podman[68648]: 2024-04-25 13:25:21.997131342 +0000 UTC m=+0.150314826 container create 84f6ae4814b46162eea167815867e82aecd11320b77d352efe38cbddcd141ba7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:25:22.225 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:22 smithi164 podman[68648]: 2024-04-25 13:25:22.224997243 +0000 UTC m=+0.378180736 container init 84f6ae4814b46162eea167815867e82aecd11320b77d352efe38cbddcd141ba7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:25:22.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:22.024+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:22.024+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:22.024+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:22.024+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:22.024+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:22.024+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:22 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:22.024+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:22.517 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:22 smithi164 podman[68648]: 2024-04-25 13:25:22.228363632 +0000 UTC m=+0.381547117 container start 84f6ae4814b46162eea167815867e82aecd11320b77d352efe38cbddcd141ba7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T13:25:22.518 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:22 smithi164 podman[68648]: 2024-04-25 13:25:22.315986515 +0000 UTC m=+0.469170016 container attach 84f6ae4814b46162eea167815867e82aecd11320b77d352efe38cbddcd141ba7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T13:25:22.807 INFO:journalctl@ceph.osd.10.smithi164.stdout:Apr 25 13:25:22 smithi164 podman[68648]: 2024-04-25 13:25:22.518312448 +0000 UTC m=+0.671495932 container died 84f6ae4814b46162eea167815867e82aecd11320b77d352efe38cbddcd141ba7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T13:25:23.128 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T13:25:23.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:23.165 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T13:25:23.165 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T13:25:23.166 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.11 2024-04-25T13:25:23.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.044+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:23.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.044+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:23.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.044+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:23.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.044+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:23.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.044+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:23.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.044+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:23.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.044+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:23.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 systemd[1]: Stopping Ceph osd.11 for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:23.981 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.480+0000 7fade009d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:23.982 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.480+0000 7fade009d700 -1 osd.11 102 *** Got signal Terminated *** 2024-04-25T13:25:23.982 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:23 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:23.480+0000 7fade009d700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:25:24.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:24.035+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:24.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:24.035+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:24.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:24.035+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:24.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:24.035+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:24.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:24.035+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:24.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:24.035+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:24.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:24 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:24.035+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:25.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:25 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:25.045+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:25.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:25 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:25.045+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:25.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:25 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:25.045+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:25.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:25 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:25.045+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:25.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:25 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:25.045+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:25.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:25 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:25.045+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:25.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:25 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:25.045+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:26.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:26 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:26.057+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:26.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:26 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:26.057+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:26.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:26 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:26.057+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:26.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:26 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:26.057+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:26.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:26 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:26.057+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:26.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:26 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:26.057+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:26.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:26 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:26.057+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:27.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:27.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:27.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:27.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:27.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:27.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:27.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:27.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:27.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:27.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:27.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:27.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:27.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:27.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:27.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:27 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:27.031+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-04-25T13:25:00.749621+0000 front 2024-04-25T13:25:00.749634+0000 (oldest deadline 2024-04-25T13:25:26.649586+0000) 2024-04-25T13:25:28.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:28.002+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-25T13:24:01.241443+0000 front 2024-04-25T13:24:01.241405+0000 (oldest deadline 2024-04-25T13:24:26.541391+0000) 2024-04-25T13:25:28.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:28.002+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-04-25T13:24:06.541915+0000 front 2024-04-25T13:24:06.542082+0000 (oldest deadline 2024-04-25T13:24:31.241845+0000) 2024-04-25T13:25:28.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:28.002+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-04-25T13:24:17.544199+0000 front 2024-04-25T13:24:17.544009+0000 (oldest deadline 2024-04-25T13:24:42.244181+0000) 2024-04-25T13:25:28.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:28.002+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-25T13:24:25.145411+0000 front 2024-04-25T13:24:25.145329+0000 (oldest deadline 2024-04-25T13:24:46.845274+0000) 2024-04-25T13:25:28.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:28.002+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-04-25T13:24:30.346327+0000 front 2024-04-25T13:24:30.346265+0000 (oldest deadline 2024-04-25T13:24:58.546674+0000) 2024-04-25T13:25:28.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:28.002+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-04-25T13:24:38.547028+0000 front 2024-04-25T13:24:38.547062+0000 (oldest deadline 2024-04-25T13:25:04.447156+0000) 2024-04-25T13:25:28.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:28.002+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-04-25T13:24:53.848730+0000 front 2024-04-25T13:24:53.848762+0000 (oldest deadline 2024-04-25T13:25:18.548588+0000) 2024-04-25T13:25:28.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:28 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11[57926]: 2024-04-25T13:25:28.002+0000 7faddc691700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-04-25T13:25:00.749621+0000 front 2024-04-25T13:25:00.749634+0000 (oldest deadline 2024-04-25T13:25:26.649586+0000) 2024-04-25T13:25:29.155 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:28 smithi193 podman[69480]: 2024-04-25 13:25:28.844147621 +0000 UTC m=+5.484646705 container died 0318ccf2853e2d4063ac18d866fb3954d2f26213cf85fce541ccea8479a813b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T13:25:29.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:29 smithi193 podman[69480]: 2024-04-25 13:25:29.155868619 +0000 UTC m=+5.796367703 container cleanup 0318ccf2853e2d4063ac18d866fb3954d2f26213cf85fce541ccea8479a813b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T13:25:29.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:29 smithi193 bash[69480]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11 2024-04-25T13:25:29.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:29 smithi193 podman[69497]: 2024-04-25 13:25:29.332371617 +0000 UTC m=+0.486202101 container remove 0318ccf2853e2d4063ac18d866fb3954d2f26213cf85fce541ccea8479a813b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T13:25:30.060 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:29 smithi193 podman[69593]: 2024-04-25 13:25:29.66857268 +0000 UTC m=+0.017949425 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:25:30.060 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:29 smithi193 podman[69593]: 2024-04-25 13:25:29.783011222 +0000 UTC m=+0.132387954 container create 82ae8caec0712f817d9d98ba6df87604f8510fcbadf56a95f854517c525e3722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:25:30.060 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:29 smithi193 podman[69593]: 2024-04-25 13:25:29.993693957 +0000 UTC m=+0.343070690 container init 82ae8caec0712f817d9d98ba6df87604f8510fcbadf56a95f854517c525e3722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T13:25:30.060 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:29 smithi193 podman[69593]: 2024-04-25 13:25:29.996828786 +0000 UTC m=+0.346205522 container start 82ae8caec0712f817d9d98ba6df87604f8510fcbadf56a95f854517c525e3722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:25:30.060 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:30 smithi193 podman[69593]: 2024-04-25 13:25:30.060234174 +0000 UTC m=+0.409610909 container attach 82ae8caec0712f817d9d98ba6df87604f8510fcbadf56a95f854517c525e3722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:25:30.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:30 smithi193 podman[69593]: 2024-04-25 13:25:30.287870308 +0000 UTC m=+0.637247042 container died 82ae8caec0712f817d9d98ba6df87604f8510fcbadf56a95f854517c525e3722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T13:25:30.836 INFO:journalctl@ceph.osd.11.smithi193.stdout:Apr 25 13:25:30 smithi193 podman[69619]: 2024-04-25 13:25:30.815494459 +0000 UTC m=+0.522571410 container remove 82ae8caec0712f817d9d98ba6df87604f8510fcbadf56a95f854517c525e3722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T13:25:30.841 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T13:25:30.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:30.877 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T13:25:30.877 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T13:25:30.877 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.a 2024-04-25T13:25:31.175 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:25:30 smithi072 systemd[1]: Stopping Ceph mds.a for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:31.524 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:25:31 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-a[71324]: 2024-04-25T13:25:31.173+0000 7f908f5d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:31.524 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:25:31 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-a[71324]: 2024-04-25T13:25:31.173+0000 7f908f5d4700 -1 mds.a *** got signal Terminated *** 2024-04-25T13:25:35.453 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:25:35 smithi072 podman[88233]: 2024-04-25 13:25:35.181504692 +0000 UTC m=+4.106049767 container died 1d514d7aff4a7def704c0748ffd393e1d0584c78d200b169bf091a91678f676c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T13:25:35.774 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:25:35 smithi072 podman[88233]: 2024-04-25 13:25:35.544276565 +0000 UTC m=+4.468821640 container cleanup 1d514d7aff4a7def704c0748ffd393e1d0584c78d200b169bf091a91678f676c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T13:25:35.774 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 25 13:25:35 smithi072 bash[88233]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-a 2024-04-25T13:25:35.971 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T13:25:36.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:36.003 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T13:25:36.003 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T13:25:36.004 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.d 2024-04-25T13:25:36.274 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:25:36 smithi072 systemd[1]: Stopping Ceph mds.d for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:36.774 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:25:36 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-d[70363]: 2024-04-25T13:25:36.333+0000 7fc7af899700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:36.774 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:25:36 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-d[70363]: 2024-04-25T13:25:36.333+0000 7fc7af899700 -1 mds.d *** got signal Terminated *** 2024-04-25T13:25:37.720 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:25:37 smithi072 podman[88355]: 2024-04-25 13:25:37.447311004 +0000 UTC m=+1.228742269 container died d4657a749527419aa839ba1c06d6b0c99f41c16ac0c67811f35ca0b90374ae8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T13:25:38.024 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:25:37 smithi072 podman[88355]: 2024-04-25 13:25:37.812256583 +0000 UTC m=+1.593687838 container cleanup d4657a749527419aa839ba1c06d6b0c99f41c16ac0c67811f35ca0b90374ae8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:25:38.024 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 25 13:25:37 smithi072 bash[88355]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-d 2024-04-25T13:25:38.242 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T13:25:38.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:38.277 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T13:25:38.277 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T13:25:38.278 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.g 2024-04-25T13:25:38.774 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:25:38 smithi072 systemd[1]: Stopping Ceph mds.g for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:38.775 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:25:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-g[70977]: 2024-04-25T13:25:38.584+0000 7f05349c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:38.775 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:25:38 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-g[70977]: 2024-04-25T13:25:38.584+0000 7f05349c8700 -1 mds.g *** got signal Terminated *** 2024-04-25T13:25:41.774 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:25:41 smithi072 podman[88486]: 2024-04-25 13:25:41.504232669 +0000 UTC m=+3.009334381 container died 5f4ecb322965451dbe35b9fae744a315d49cc3209b7705fd6ea73ebce5c8f5b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:25:42.166 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:25:41 smithi072 podman[88486]: 2024-04-25 13:25:41.862468946 +0000 UTC m=+3.367570657 container cleanup 5f4ecb322965451dbe35b9fae744a315d49cc3209b7705fd6ea73ebce5c8f5b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T13:25:42.166 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 25 13:25:41 smithi072 bash[88486]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-g 2024-04-25T13:25:42.329 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T13:25:42.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:42.361 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T13:25:42.361 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T13:25:42.361 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.j 2024-04-25T13:25:42.774 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:42 smithi072 systemd[1]: Stopping Ceph mds.j for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:42.775 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:42 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j[70013]: 2024-04-25T13:25:42.666+0000 7f926955b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:42.775 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:42 smithi072 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j[70013]: 2024-04-25T13:25:42.666+0000 7f926955b700 -1 mds.j *** got signal Terminated *** 2024-04-25T13:25:47.022 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:46 smithi072 podman[88607]: 2024-04-25 13:25:46.679221988 +0000 UTC m=+4.105622533 container died a009131c2d33c2f50d2c10846c50fda0a30f7240d080a9d02842b059a5c412d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T13:25:47.274 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:47 smithi072 podman[88607]: 2024-04-25 13:25:47.112894915 +0000 UTC m=+4.539295458 container cleanup a009131c2d33c2f50d2c10846c50fda0a30f7240d080a9d02842b059a5c412d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T13:25:47.275 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:47 smithi072 bash[88607]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j 2024-04-25T13:25:47.591 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T13:25:47.612 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:47 smithi072 podman[88621]: 2024-04-25 13:25:47.348194651 +0000 UTC m=+0.668583432 container remove a009131c2d33c2f50d2c10846c50fda0a30f7240d080a9d02842b059a5c412d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T13:25:47.612 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:47 smithi072 systemd[1]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.j.service: Deactivated successfully. 2024-04-25T13:25:47.612 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:47 smithi072 systemd[1]: Stopped Ceph mds.j for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:25:47.612 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 25 13:25:47 smithi072 systemd[1]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.j.service: Consumed 1.070s CPU time. 2024-04-25T13:25:47.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:47.623 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T13:25:47.623 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T13:25:47.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.b 2024-04-25T13:25:48.003 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:25:47 smithi164 systemd[1]: Stopping Ceph mds.b for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:48.306 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:25:48 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b[62256]: 2024-04-25T13:25:48.001+0000 7f59b5d6c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:48.306 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:25:48 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b[62256]: 2024-04-25T13:25:48.001+0000 7f59b5d6c700 -1 mds.b *** got signal Terminated *** 2024-04-25T13:25:50.808 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:25:50 smithi164 podman[68752]: 2024-04-25 13:25:50.498169822 +0000 UTC m=+2.680878743 container died 32fe3eaa671778e827f4da3e299ee5eef9dc4772c2c3566503f264ca82a88acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:25:51.427 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:25:51 smithi164 podman[68752]: 2024-04-25 13:25:51.177105401 +0000 UTC m=+3.359814312 container cleanup 32fe3eaa671778e827f4da3e299ee5eef9dc4772c2c3566503f264ca82a88acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T13:25:51.428 INFO:journalctl@ceph.mds.b.smithi164.stdout:Apr 25 13:25:51 smithi164 bash[68752]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-b 2024-04-25T13:25:51.548 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T13:25:51.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:51.583 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T13:25:51.583 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T13:25:51.583 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.e 2024-04-25T13:25:51.807 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:25:51 smithi164 systemd[1]: Stopping Ceph mds.e for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:52.307 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:25:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e[61620]: 2024-04-25T13:25:51.880+0000 7f1eaedcd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:52.307 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:25:51 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e[61620]: 2024-04-25T13:25:51.880+0000 7f1eaedcd700 -1 mds.e *** got signal Terminated *** 2024-04-25T13:25:54.772 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:25:54 smithi164 podman[68885]: 2024-04-25 13:25:54.499206923 +0000 UTC m=+2.694388184 container died 857c802927013e9e87f8c091b132e634399336e7206af14dd217c89a1b997ccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T13:25:55.057 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:25:54 smithi164 podman[68885]: 2024-04-25 13:25:54.872123397 +0000 UTC m=+3.067304664 container cleanup 857c802927013e9e87f8c091b132e634399336e7206af14dd217c89a1b997ccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T13:25:55.057 INFO:journalctl@ceph.mds.e.smithi164.stdout:Apr 25 13:25:54 smithi164 bash[68885]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-e 2024-04-25T13:25:55.299 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T13:25:55.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:55.333 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T13:25:55.333 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T13:25:55.333 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.h 2024-04-25T13:25:55.557 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:25:55 smithi164 systemd[1]: Stopping Ceph mds.h for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:25:56.057 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:25:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h[61931]: 2024-04-25T13:25:55.633+0000 7f4f41577700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:25:56.057 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:25:55 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h[61931]: 2024-04-25T13:25:55.633+0000 7f4f41577700 -1 mds.h *** got signal Terminated *** 2024-04-25T13:25:58.807 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:25:58 smithi164 podman[69005]: 2024-04-25 13:25:58.502179307 +0000 UTC m=+2.952603260 container died 9d9ba379a90a53476084f24bbd217392fb5ac298cefa48cbd8ff43cd05ac2f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T13:25:59.136 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:25:58 smithi164 podman[69005]: 2024-04-25 13:25:58.942271475 +0000 UTC m=+3.392695467 container cleanup 9d9ba379a90a53476084f24bbd217392fb5ac298cefa48cbd8ff43cd05ac2f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:25:59.137 INFO:journalctl@ceph.mds.h.smithi164.stdout:Apr 25 13:25:58 smithi164 bash[69005]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-h 2024-04-25T13:25:59.361 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T13:25:59.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:25:59.393 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T13:25:59.393 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T13:25:59.393 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.k 2024-04-25T13:25:59.687 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:25:59 smithi164 systemd[1]: Stopping Ceph mds.k for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:26:00.057 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:25:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k[61302]: 2024-04-25T13:25:59.686+0000 7fd0ac676700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:26:00.057 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:25:59 smithi164 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k[61302]: 2024-04-25T13:25:59.686+0000 7fd0ac676700 -1 mds.k *** got signal Terminated *** 2024-04-25T13:26:04.012 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:26:03 smithi164 podman[69127]: 2024-04-25 13:26:03.696588621 +0000 UTC m=+4.091385410 container died 4ac470d933d855c9a11070aae3a5310303e0fadcbbee84c4f9a46e79126a6d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T13:26:04.287 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:26:04 smithi164 podman[69127]: 2024-04-25 13:26:04.111968299 +0000 UTC m=+4.506765095 container cleanup 4ac470d933d855c9a11070aae3a5310303e0fadcbbee84c4f9a46e79126a6d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T13:26:04.287 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:26:04 smithi164 bash[69127]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k 2024-04-25T13:26:04.537 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:26:04 smithi164 podman[69141]: 2024-04-25 13:26:04.28770525 +0000 UTC m=+0.587176277 container remove 4ac470d933d855c9a11070aae3a5310303e0fadcbbee84c4f9a46e79126a6d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:26:04.538 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:26:04 smithi164 systemd[1]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-25T13:26:04.538 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:26:04 smithi164 systemd[1]: Stopped Ceph mds.k for d8ca43ea-0303-11ef-bc93-c7b262605968. 2024-04-25T13:26:04.538 INFO:journalctl@ceph.mds.k.smithi164.stdout:Apr 25 13:26:04 smithi164 systemd[1]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.092s CPU time. 2024-04-25T13:26:04.539 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T13:26:04.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:26:04.570 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T13:26:04.571 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T13:26:04.571 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.c 2024-04-25T13:26:04.847 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:26:04 smithi193 systemd[1]: Stopping Ceph mds.c for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:26:05.113 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:26:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c[62817]: 2024-04-25T13:26:04.846+0000 7fabdcf05700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:26:05.114 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:26:04 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c[62817]: 2024-04-25T13:26:04.847+0000 7fabdcf05700 -1 mds.c *** got signal Terminated *** 2024-04-25T13:26:06.756 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:26:06 smithi193 podman[69697]: 2024-04-25 13:26:06.485314415 +0000 UTC m=+1.718428704 container died d985942e3e7cdd4f728d4c4b9cc9964ae8b4bebf996cfb2750514c845ca6941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T13:26:07.098 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:26:06 smithi193 podman[69697]: 2024-04-25 13:26:06.855902672 +0000 UTC m=+2.089016982 container cleanup d985942e3e7cdd4f728d4c4b9cc9964ae8b4bebf996cfb2750514c845ca6941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:26:07.099 INFO:journalctl@ceph.mds.c.smithi193.stdout:Apr 25 13:26:06 smithi193 bash[69697]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-c 2024-04-25T13:26:07.227 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T13:26:07.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:26:07.263 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T13:26:07.264 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T13:26:07.264 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.f 2024-04-25T13:26:07.481 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:26:07 smithi193 systemd[1]: Stopping Ceph mds.f for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:26:07.982 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:26:07 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f[62501]: 2024-04-25T13:26:07.584+0000 7f6d96887700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:26:07.982 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:26:07 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f[62501]: 2024-04-25T13:26:07.584+0000 7f6d96887700 -1 mds.f *** got signal Terminated *** 2024-04-25T13:26:10.793 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:26:10 smithi193 podman[69832]: 2024-04-25 13:26:10.495007337 +0000 UTC m=+3.010369044 container died 9aab990b038b1e5ba411bd8cf6776274110ee636ee05079ecafd2d7224ee4fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:26:10.793 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:26:10 smithi193 podman[69832]: 2024-04-25 13:26:10.793172882 +0000 UTC m=+3.308534588 container cleanup 9aab990b038b1e5ba411bd8cf6776274110ee636ee05079ecafd2d7224ee4fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:26:11.148 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:26:10 smithi193 bash[69832]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f 2024-04-25T13:26:11.149 INFO:journalctl@ceph.mds.f.smithi193.stdout:Apr 25 13:26:10 smithi193 podman[69845]: 2024-04-25 13:26:10.944430637 +0000 UTC m=+0.447930156 container remove 9aab990b038b1e5ba411bd8cf6776274110ee636ee05079ecafd2d7224ee4fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T13:26:11.167 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T13:26:11.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:26:11.199 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T13:26:11.199 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T13:26:11.199 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.i 2024-04-25T13:26:11.481 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:26:11 smithi193 systemd[1]: Stopping Ceph mds.i for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:26:11.981 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:26:11 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i[63140]: 2024-04-25T13:26:11.529+0000 7fb080c44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:26:11.982 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:26:11 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i[63140]: 2024-04-25T13:26:11.529+0000 7fb080c44700 -1 mds.i *** got signal Terminated *** 2024-04-25T13:26:14.764 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:26:14 smithi193 podman[69954]: 2024-04-25 13:26:14.491094346 +0000 UTC m=+3.075581026 container died ae363537a1acbe61378b2365269e38574c6ad377f2864e82378b1016127c55db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T13:26:15.098 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:26:14 smithi193 podman[69954]: 2024-04-25 13:26:14.846708191 +0000 UTC m=+3.431194878 container cleanup ae363537a1acbe61378b2365269e38574c6ad377f2864e82378b1016127c55db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T13:26:15.098 INFO:journalctl@ceph.mds.i.smithi193.stdout:Apr 25 13:26:14 smithi193 bash[69954]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-i 2024-04-25T13:26:15.218 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T13:26:15.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:26:15.250 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T13:26:15.250 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T13:26:15.250 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.l 2024-04-25T13:26:15.482 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:26:15 smithi193 systemd[1]: Stopping Ceph mds.l for d8ca43ea-0303-11ef-bc93-c7b262605968... 2024-04-25T13:26:15.981 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:26:15 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l[62196]: 2024-04-25T13:26:15.592+0000 7fe7df004700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:26:15.982 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:26:15 smithi193 ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l[62196]: 2024-04-25T13:26:15.592+0000 7fe7df004700 -1 mds.l *** got signal Terminated *** 2024-04-25T13:26:17.626 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:26:17 smithi193 podman[70074]: 2024-04-25 13:26:17.333082638 +0000 UTC m=+1.867049417 container died 26563088b66f20a5ac9a83b2fe72db5b4aa3f3de95349d28b883e9fe1d3f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T13:26:17.981 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:26:17 smithi193 podman[70074]: 2024-04-25 13:26:17.701188685 +0000 UTC m=+2.235155457 container cleanup 26563088b66f20a5ac9a83b2fe72db5b4aa3f3de95349d28b883e9fe1d3f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T13:26:17.982 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:26:17 smithi193 bash[70074]: ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l 2024-04-25T13:26:17.982 INFO:journalctl@ceph.mds.l.smithi193.stdout:Apr 25 13:26:17 smithi193 podman[70086]: 2024-04-25 13:26:17.84430917 +0000 UTC m=+0.514008176 container remove 26563088b66f20a5ac9a83b2fe72db5b4aa3f3de95349d28b883e9fe1d3f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d8ca43ea-0303-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T13:26:18.058 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8ca43ea-0303-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T13:26:18.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:26:18.088 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T13:26:18.088 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:26:18.279 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:26:20.611 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:26:20.802 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:26:23.295 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:26:23.483 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:26:25.810 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:25.842 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:25.875 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:25.900 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T13:26:25.902 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/remote/smithi072/crash 2024-04-25T13:26:25.903 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:26:25.948 INFO:teuthology.orchestra.run.smithi072.stderr:tar: /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:26:25.948 INFO:teuthology.orchestra.run.smithi072.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:26:25.950 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/remote/smithi164/crash 2024-04-25T13:26:25.951 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:26:25.990 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:26:25.990 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:26:25.992 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/remote/smithi193/crash 2024-04-25T13:26:25.993 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:26:26.027 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:26:26.028 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:26:26.029 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T13:26:26.029 DEBUG:teuthology.orchestra.run.smithi072:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T13:26:26.064 INFO:tasks.cephadm:Compressing logs... 2024-04-25T13:26:26.065 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:26:26.107 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:26:26.110 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:26:26.129 INFO:teuthology.orchestra.run.smithi072.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:26:26.129 INFO:teuthology.orchestra.run.smithi072.stderr:: No such file or directory 2024-04-25T13:26:26.130 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:26:26.130 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:26:26.133 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T13:26:26.133 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T13:26:26.133 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T13:26:26.133 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T13:26:26.133 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T13:26:26.133 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T13:26:26.134 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mgr.x.log: 87.9% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:26:26.134 INFO:teuthology.orchestra.run.smithi072.stderr: 90.3% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:26:26.148 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:26:26.148 INFO:teuthology.orchestra.run.smithi164.stderr:: No such file or directory 2024-04-25T13:26:26.149 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:26:26.149 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:26:26.149 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T13:26:26.149 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T13:26:26.149 INFO:teuthology.orchestra.run.smithi164.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:26:26.149 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T13:26:26.149 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.7.log: /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T13:26:26.149 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T13:26:26.150 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:26:26.150 INFO:teuthology.orchestra.run.smithi164.stderr: -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T13:26:26.150 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T13:26:26.150 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:26:26.150 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.10.log: 90.5% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:26:26.150 INFO:teuthology.orchestra.run.smithi164.stderr: 91.8% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T13:26:26.152 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T13:26:26.152 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:26:26.153 INFO:teuthology.orchestra.run.smithi072.stderr: 94.5%/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.6.log: -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T13:26:26.153 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T13:26:26.153 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.cephadm.log: 94.6% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T13:26:26.153 INFO:teuthology.orchestra.run.smithi072.stderr: 84.2% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:26:26.153 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:26:26.153 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T13:26:26.153 INFO:teuthology.orchestra.run.smithi072.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:26:26.153 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T13:26:26.154 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T13:26:26.154 INFO:teuthology.orchestra.run.smithi072.stderr: 94.5% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T13:26:26.154 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:26:26.155 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:26:26.155 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-04-25T13:26:26.155 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:26:26.155 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T13:26:26.155 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T13:26:26.155 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:26:26.155 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T13:26:26.156 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.audit.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gzgzip 2024-04-25T13:26:26.156 INFO:teuthology.orchestra.run.smithi193.stderr: -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T13:26:26.156 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T13:26:26.156 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T13:26:26.156 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T13:26:26.156 INFO:teuthology.orchestra.run.smithi193.stderr: 88.0% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:26:26.156 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.5% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:26:26.156 INFO:teuthology.orchestra.run.smithi193.stderr: 91.9% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T13:26:26.157 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:26:26.157 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T13:26:26.157 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T13:26:26.157 INFO:teuthology.orchestra.run.smithi193.stderr: 83.9% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:26:26.159 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T13:26:26.159 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:26:26.159 INFO:teuthology.orchestra.run.smithi164.stderr: 83.9% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:26:26.159 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T13:26:26.160 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:26:26.161 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T13:26:26.167 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:26:26.173 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T13:26:26.177 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T13:26:26.189 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.l.log: 93.4% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T13:26:26.190 INFO:teuthology.orchestra.run.smithi193.stderr: 94.5% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T13:26:26.193 INFO:teuthology.orchestra.run.smithi193.stderr: 94.6% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T13:26:26.194 INFO:teuthology.orchestra.run.smithi193.stderr: 91.0% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:26:26.202 INFO:teuthology.orchestra.run.smithi193.stderr: 93.2% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T13:26:26.209 INFO:teuthology.orchestra.run.smithi164.stderr: 94.0% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T13:26:26.217 INFO:teuthology.orchestra.run.smithi164.stderr: 94.4% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T13:26:26.295 INFO:teuthology.orchestra.run.smithi164.stderr: 93.5% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T13:26:26.457 INFO:teuthology.orchestra.run.smithi164.stderr: 92.1% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T13:26:26.477 INFO:teuthology.orchestra.run.smithi072.stderr: 90.2% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T13:26:26.498 INFO:teuthology.orchestra.run.smithi193.stderr: 92.0% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T13:26:26.547 INFO:teuthology.orchestra.run.smithi193.stderr: 93.3% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T13:26:26.567 INFO:teuthology.orchestra.run.smithi072.stderr: 92.6% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T13:26:26.592 INFO:teuthology.orchestra.run.smithi193.stderr: 92.8% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T13:26:26.593 INFO:teuthology.orchestra.run.smithi164.stderr: 92.9% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T13:26:26.619 INFO:teuthology.orchestra.run.smithi072.stderr: 92.8% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T13:26:26.666 INFO:teuthology.orchestra.run.smithi193.stderr: 92.7% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T13:26:26.673 INFO:teuthology.orchestra.run.smithi072.stderr: 92.9% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T13:26:26.694 INFO:teuthology.orchestra.run.smithi164.stderr: 92.6% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T13:26:26.698 INFO:teuthology.orchestra.run.smithi164.stderr: 92.8% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T13:26:26.737 INFO:teuthology.orchestra.run.smithi193.stderr: 92.6% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T13:26:26.739 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-04-25T13:26:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:real 0m0.615s 2024-04-25T13:26:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:user 0m2.430s 2024-04-25T13:26:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m0.110s 2024-04-25T13:26:26.756 INFO:teuthology.orchestra.run.smithi072.stderr: 93.0% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T13:26:26.783 INFO:teuthology.orchestra.run.smithi164.stderr: 92.8% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T13:26:26.785 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-25T13:26:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m0.662s 2024-04-25T13:26:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m2.747s 2024-04-25T13:26:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.118s 2024-04-25T13:26:26.986 INFO:teuthology.orchestra.run.smithi072.stderr: 90.5% -- replaced with /var/log/ceph/d8ca43ea-0303-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T13:26:26.988 INFO:teuthology.orchestra.run.smithi072.stderr: 2024-04-25T13:26:26.988 INFO:teuthology.orchestra.run.smithi072.stderr:real 0m0.868s 2024-04-25T13:26:26.988 INFO:teuthology.orchestra.run.smithi072.stderr:user 0m3.217s 2024-04-25T13:26:26.988 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m0.127s 2024-04-25T13:26:26.988 INFO:tasks.cephadm:Archiving logs... 2024-04-25T13:26:26.989 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/remote/smithi072/log 2024-04-25T13:26:26.990 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:26:27.502 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/remote/smithi164/log 2024-04-25T13:26:27.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:26:27.905 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/remote/smithi193/log 2024-04-25T13:26:27.906 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:26:28.278 INFO:tasks.cephadm:Removing cluster... 2024-04-25T13:26:28.278 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 --force 2024-04-25T13:26:28.466 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:26:28.710 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 --force 2024-04-25T13:26:28.902 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:26:29.161 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8ca43ea-0303-11ef-bc93-c7b262605968 --force 2024-04-25T13:26:29.349 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: d8ca43ea-0303-11ef-bc93-c7b262605968 2024-04-25T13:26:29.604 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T13:26:29.605 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:26:29.620 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:26:29.636 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:26:29.651 INFO:tasks.cephadm:Teardown complete 2024-04-25T13:26:29.651 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T13:26:29.663 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T13:26:29.663 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:26:29.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:26:29.679 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:26:29.794 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T13:26:29.807 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T13:26:29.807 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:26:29.809 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:26:29.812 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:26:29.821 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:26:29.823 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:26:29.825 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:26:29.825 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:26:29.826 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-04-25T13:26:29.826 INFO:teuthology.orchestra.run.smithi072.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 +917us[ +933us] +/- 56ms 2024-04-25T13:26:29.826 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -1584us[-1584us] +/- 64ms 2024-04-25T13:26:29.826 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 61 +636us[ +652us] +/- 72ms 2024-04-25T13:26:29.826 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:26:29.828 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:26:29.828 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-25T13:26:29.828 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +919us[ +923us] +/- 56ms 2024-04-25T13:26:29.828 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -1581us[-1581us] +/- 64ms 2024-04-25T13:26:29.828 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +598us[ +598us] +/- 72ms 2024-04-25T13:26:29.828 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:26:29.830 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:26:29.830 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-04-25T13:26:29.830 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 7 377 127 +876us[ +873us] +/- 55ms 2024-04-25T13:26:29.830 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -1627us[-1627us] +/- 64ms 2024-04-25T13:26:29.830 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 127 +589us[ +589us] +/- 71ms 2024-04-25T13:26:29.831 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:26:29.831 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T13:26:29.842 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T13:26:29.843 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T13:26:29.855 DEBUG:teuthology.orchestra.run.smithi072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:26:29.938 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:26:30.004 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:26:30.068 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:26:30.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:26:30.096 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:26:30.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:26:30.123 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:26:30.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:26:30.150 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T13:26:30.161 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T13:26:30.203 INFO:teuthology.task.internal:Duration was 1918.353611 seconds 2024-04-25T13:26:30.204 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T13:26:30.216 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T13:26:30.216 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:26:30.218 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:26:30.221 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:26:30.253 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:26:30.254 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:26:30.256 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:26:30.596 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T13:26:30.597 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2024-04-25T13:26:30.597 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:26:30.622 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-04-25T13:26:30.622 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:26:30.665 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T13:26:30.666 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:26:30.696 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T13:26:30.697 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:26:30.699 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:26:30.708 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:26:30.761 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T13:26:30.761 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:26:30.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:26:30.774 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:26:31.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T13:26:31.138 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T13:26:31.138 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:26:31.162 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:26:31.186 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:26:31.210 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T13:26:31.223 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:26:31.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:26:31.229 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:26:31.245 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2024-04-25T13:26:31.248 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-04-25T13:26:31.272 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-04-25T13:26:31.292 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:26:31.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:26:31.311 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:26:31.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:26:31.346 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:26:31.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:26:31.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T13:26:31.374 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T13:26:31.375 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/remote/smithi072 2024-04-25T13:26:31.375 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:26:31.414 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/remote/smithi164 2024-04-25T13:26:31.414 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:26:31.450 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460/remote/smithi193 2024-04-25T13:26:31.450 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:26:31.484 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T13:26:31.484 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:26:31.486 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:26:31.489 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:26:31.532 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T13:26:31.548 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T13:26:31.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T13:26:31.631 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T13:26:31.632 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:26:31.634 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:26:31.636 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:26:31.646 INFO:teuthology.orchestra.run.smithi072.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:26 /home/ubuntu/cephtest 2024-04-25T13:26:31.648 INFO:teuthology.orchestra.run.smithi164.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:26 /home/ubuntu/cephtest 2024-04-25T13:26:31.650 INFO:teuthology.orchestra.run.smithi193.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:26 /home/ubuntu/cephtest 2024-04-25T13:26:31.651 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T13:26:31.663 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T13:26:31.689 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T13:26:31.715 DEBUG:teuthology.nuke:shortname: smithi072 2024-04-25T13:26:31.715 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:26:31.743 DEBUG:teuthology.nuke:shortname: smithi164 2024-04-25T13:26:31.744 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:26:31.749 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:48:13.842521', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxP6xRdHtzJ2LIVS0OKQ9cP2mrxnzNrXZ753KSRnC5JBbBgyBSE2Dp7PfQiD3aU4KGSHe43+lofbxGK14ZbHUc='} 2024-04-25T13:26:31.768 DEBUG:teuthology.nuke:shortname: smithi193 2024-04-25T13:26:31.768 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:26:31.772 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:48:13.845076', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2sD3CpQLsQ+jhRrT4LV86EUUW+/xgHzg5nZ91wd7EZfVT+lsVTpjRHovdL9JP3p42aRAExaXJ/jI43Cd3m0xk='} 2024-04-25T13:26:31.794 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673460', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:48:13.843879', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPADk437WhqxgifqptXf8jpu+bdW+KzofJvFtlOiJKYJ3ZLkZwomBWg8sg1n0cRr1Y6ydR3HmHQsgPObe8ByaL0='} 2024-04-25T13:26:31.805 INFO:teuthology.orchestra.console.smithi072:Power off 2024-04-25T13:26:31.805 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:26:31.832 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2024-04-25T13:26:31.833 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:26:31.850 INFO:teuthology.orchestra.console.smithi164:Power off 2024-04-25T13:26:31.850 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:26:31.864 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-04-25T13:26:31.874 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-04-25T13:26:31.874 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:26:31.891 INFO:teuthology.orchestra.console.smithi193:Power off 2024-04-25T13:26:31.892 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:26:31.902 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-25T13:26:31.915 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off 2024-04-25T13:26:31.915 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:26:31.947 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-25T13:26:35.865 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:26:35.902 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:26:35.949 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:26:35.976 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-04-25T13:26:36.014 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-25T13:26:36.059 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-25T13:26:39.978 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:26:40.014 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:26:40.061 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:26:40.088 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-04-25T13:26:40.124 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-04-25T13:26:40.171 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off 2024-04-25T13:26:40.189 INFO:teuthology.orchestra.console.smithi072:Power off completed 2024-04-25T13:26:40.225 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-04-25T13:26:40.272 INFO:teuthology.orchestra.console.smithi193:Power off completed 2024-04-25T13:26:40.378 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 1918.3536105155945 failure_reason: '''NoneType'' object has no attribute ''get''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=e6b72320fc8240599e9357b283b792e8 status: fail success: false 2024-04-25T13:26:40.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:26:40.478 INFO:teuthology.run:FAIL